DOCOHENT BBSOHB 



BD 124 188 



IB 003 581 



AOTHOR 
TITLE 



FOB D&TE « 
NOTE I 



EDPS PHICE 
DESCRIPTORS 



IDENTIFIERS 



Hill, Robert Thoias- 

The Developient and I Bplenentatlon of a Model for 
Adainistering a Vlsaal Test 'of Achievement Over 
Broadcast Television. _ 
May 76 

95p.; Master's thesis, Pennsylvemia ^tate 
University 

MF-$0.93 HG-$tt.67 Plus Postage. 
"•"Achieveient . Tests; ♦Broadca^;st Television; 
♦Educational Television; Eleientaty Education; 
♦Elementary School Science; Masters Theses; Hod^s; 
Science Instruction; ♦Testing , • , 

Pennsylvania; ♦Project Science for \he Seventies; 
Television Test of Science Processes ^ 



ABSTRACT . ^ ^ 

One component of Pennsylvania's Science for the 
Seventies Project^ an experimental science ctirrlcului for elementary 
schools vhich makes extensive use of instructional tele vision, was an 
^evaluation vhich compared the effectiveness of the project to that of 
.existing, instructional programs. As part of the evaluation, a model 
'was developed and implemented for administering the Television Test 
of Sciences Processes 'via public television broadcast to 3,500 
geographically dispersed students. The tests were successfully 
administered, and the results suggest that the model can.be used as a 
prototy|)e for the administration of other examinations.- This text ^ 
provides Background information^ a review of relevant literature, a • 
description of the model design and implementation, a summary of 
results, and further recommendations. (EHH) 



♦ Documents^ acquired l^y ERIC include mkny informal unpublisjbed ♦ 

♦ materials not. available from' othei^ sources. ERIC makes every effort ♦ 

♦ to obtain' the best copy available. Neverthel€Jss, items of marg4n?tl ♦ 

♦ reproducibility are often encountered' and this affects the quality ♦ 

♦ of the microfiche and hardcopy reproductions ERIC make^ available ♦ 

♦ via the ERIC Docujnent Reproduction Service (EDBS) . EliRS is not * 

♦ responsible jtoz the quality of the origiiaal document. Reproductions ♦ 

♦ supplied by EDRS are the best that can be made from the original. ♦ 



The Pennaylvanla Stat^ University 
* The Graduate School 

Division of Academi4 Curriculum and Instruction* 



The Deve].opment and Implementation of a 
Model/ for Adnilnlsterlng a Visual Test of 
Achievement Over -Brqadcast Television 



* OiPARTMItlTqPHiAtTH. 
■ OUCATIONAWILFAIII 
NATiONALiNSTlTOriOP 

THIS OPCUMENT HAS BEEN ftmlmn 
eOUCATi^ON POSIT.ON OR POLICY 



A Thesis In 
i^ademlc 'CurrlcCilum and Instruction 
by 

Robert Tho&as ^Hlll 



Submitted In partial fulfillment 
of the Requirements 
for the, Degree \pf , 



•1 



Mascer /of Sclencj 



ly 1976 



Date 'of Approval: 



s 



latoirles: 

Paul W. Welliver, Professor of 
Education 
Thesis Advisor 




Carol A. Cartvrright 
In Charge of Graduate Pifograms 
Division of Academic Curriculum 
and .Instruction 



j . ' ' ^ ACKNOWLEDGEMEHTS . 

Deep and sincere 'appreciation Is extended to the unnamed 
students » classtoom teachers /and school^adiplnlstr^tors whose 

^ ^ ^ . ■ . ■ . r 

cooperation » Interest » and support have m^de th4 coiQp^etlon o^ this 
study possible. ^ . ' a A i . 

The writer Is Indebted to William BarKihart» William Williams^ 
Nand'^Geraldlne Reed of the Alim^eny Educational Btoadcast Council » 
aM of whom made valuable contributions both In the planning and 
execution stages. In addition, special mention Is made' of the 
achninlstratlve support vWlllIngly g^lven by Nomia Frontz and, 
Sher^N^atsavakls and the guidance and assistance rendered by David 

TorrenceV developer of the Television Test of Science Processes. 

\ . ^ * ■ • 

To ade^ately express the gratitude owed Dr. Pau^ Welliv^r» 
thesl^* advisor » director of the "Science for the Seventies"/ \ 
Project* and friend, would require more than the space of this page. 
Suffice It to say that his advice, spggestlonst Insight and ; ^ 

dedication wete Instrumental In the completion of this study. 



iii 




TABLE OF CONTENTS 

, ^ ' '% ^ \ 

'acknowledgements 

LJST OF TABLES ....... ^ ^ . . . 

LIST OF FIGURES.. * . yl 

' ' ' . ■ ' ^ ' 

CHAPTER I: INTRODUCTION \ 

' General Introduction. 1 

Background. - 2 

Need' for the Study. . \ . ^ ^ ....... . & 

Statement of the Probletm. ........ J . 7 

Significance of the. Study .\ 7 

Definition of Terms .......... 8 

Summary .^ .> 9 

CHAPTER II: REVIEW OF THE LITERATURE AND RELAtED STUDIES 

Method. . 10 

Visual Testing. .............. 10 

Test Administration Via Television 12 

Testing by Television • • 15 

Summary ,i . 19 

CHAPTER III: PROCEDURES AND DESIGN ^ 

Introduction. u ..... ^. 21 

Analysis of. the Problem '....>., 21 

Development of ^the" Model 23 

Design for Impl^entatlon o^ the- Model to the 

Administration of the TTSP. .... \' . 25 

Stage I: Identification of Test Population ... ... . . 26 

" Stage II: Implementation of Logistical System'. ..... 28 

Stage III: Eva:i.uatlon of Testing Procedures. ...... 31 

* Suipm^ty* . : • • • 33 



CHAPTER IV: RBfiULTS OF THE IMPLEMENTATION OF THE MOD^L | 

Introductlo^-. ........ ^ .. 34 

Stage I: IdNeN^f Icatlon of Test Population^. ...... |f 

Stage II: Implenientatlon of Logistical System. fi. 

Stage III: Evalu^lon of- Testing Procedures. ....... 41 

Suiimary . . .^|r*?^ . ' 49 



CHAPTER 



V: SUMMARY, FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS: 



'^Stiitement of the Problem.. > . 50 

Suranary . 50 



Presentation and Dlsciis/ion of the Findings ....... 51 

ConclfU^lons (.........•»........ 56 , 

Guidelines £or Implementing the Model 57 

^commendations for Further Study ... • • . . \ ■'' • . 62 ^ 

B^^BLIOGH^HY 1, * 64 

APPENDIX A: TELEVISION TEST OF SCIENCE PROCESSES TEST , 

BOOKLET V • • • • 67 

\, . 

APP^IX b': television TEST OF SCIENCE PROCESSES/ 

\\ TEACHER'S TEST MANUAL 79 



APPENDIX C: LETTER FROM WILLIAM M. BARNHART, AEBC, TO • 
SUPERINTENDENTS, MEMBER DISTRICTS OF AEBC, AND 
AEBC DISTRICT DESIGNATES, DATED AUGUST 14, 1975 . ' 

APPENDIX D: LETTER PROM DAVID TORRENCE, TEST DEVELOPER 

TELEVISION TEST OF SCIENCE "PROCESSES TO AR^- - 
TEST gOORDINATORS, . DATED DECEMBER 15, .4.975. . 



... -x 





82 



85 



Table 



LIST OF TABLES 



\ 



Page 



1 School Districts and Schools Participating in the 

TTSP V ,i . . 38 

2 ' Tabulated Results> of Tedcher'a Survey, Television 

Test of Science Prpcesses./.; ..... .^j. ... '. .I . * 




• I, 



0 



4. . 



vl 



LIST OF FIGURES 



Figure 
1 

2 
3 

6 • 
5 



: ^ * . Page 

Model for the' Administration^ of a Visual Test, of 

. Achievement Udlng Broadcast televl^foa^ as a 

Delivery Medium. . . -^^^ • • . - • T . . • , 

The Model, Stage I ^ . F ............. 26 

The Model, Stage II. . . . •.. ........ ^ . 29 

The Model, Stage III .... . ........ 31 

.Geographical Distribution of PartlQlpating Schobl 
Districts and Schools. ... v ^. 



7 



7 



CHAPTER I 



INTRODUCTION 



General Introduction 



\ 



Television has become firmly established as a valuable tool .for 
educators. Its uses range, from something as simple and relatively 
inexpensive as videotaping the school football team practice on a 
backpack racotder with a hand-held camera to the sophisticated, 
complex and deQddedly expensiye productions broadcast over the .public 
television Qett^ork* Many academic disciplines at all educational 
levels have resorted to the f lexibil;Lt:y , convenience, and impact that 
television can bring to the classroom. ^ , ■ , \ ■ 

Goodwin Chu and 'Wilbur Schramm (19^7) in tKeir review of 
instructional (television researc h have come to this conclusipn; 




For one thing, it has>become cl^ar that there is . ' 
no longer any. reason to raise the question 'whether 
instructional television, can servjfe as an efficient * 
tool of learning. . . the evidemce/is now Overwhelming 
that it can and, -under favorable circumstances., does. 

Thus, those involved in exploring how television can l>est function 

as an instructioxial device need npt l^e concerned witft justifying'^its 

use in education. Its widespread acceptance and the scientific vali- 

(Uation Of its ef fectivertess have put that issue to rest. Rather, the 

i ; • ' • • 

Questions now addressings* teachers," instructional developers, and 

"^faiedia specialists fall into two major categories. They are, first, 

^how can the quality of instructional^ television be improved., andf 

second,' how c^n the us^ of instriictjLpnal television be expanded to 

/ improve in&fetuGtlon? ' 
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The ilnes between these two categories are not ^rlgld and 
distinct, that is, improvement and expansion are interwoven. They 
affect each other, for an expansion without quality will not be 
accepted er tolerated by educators. In the same w^y,^ the quality of 
instructional television can be« enhanced by a broadening of its 
applications. 

This study will have its emphasis on the second category of 
expansion. It will bte concerned with the relatively unexplored area, 
of^ using television as a medium fpr achievement testing.* 

Backfground " \ 

The origins of this study begin with the "Science for the. 
Seventies" Project coriducted under the supervision of the 
Pennsylvania Department of Education from 1,968 to the present. The 

purpose of this proje<|£ has been to design, develop. Implement, and^, 

I ■ ■ I / 

evaluate a program of elementary school science which emphasizes tht} 

more recently developed strategies for teaching science. The result 

•has been a program of instruction entitled "Science for the Seventies 

(SFTS) whicjj makes extensive use of instructional television lesson^ 

in combination with coordinated classrobra'v activities. 

In conjunction with this project, a^, television test has been 
produced to assess the comprehension of science processes in 
elementary school children. There were three main reasons for the 
development of this test. < 

First\jj there was a general need for a way t6 measure the effect 
the SFTS instructional program had in comparison with existing 
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programs. The SFTS developers were emphatic in this necessity. : 

With all of these resources, there will also be 
'a means of evafuating the status of the elementary 

school science program in a particular school or , u 
. school district. This evaluative procedure will . • 

be capable of assessing a number of different 
facets of the scienC^ program.^ 

A school or school district desiring assistance 
in imj^roving its sciencie program could then 
utilize this evaluative procedure to identify 
its strengths and weaknesses. This diagnosis 
could be used as a basis for prescribing a 
number of alternatives of re^urces which could 
be directed toward overcoming the deficiencies 
which were identified. y<SFTS and Instructional 
Television, Phase Il^/^oject Report, 1975^ p» 2). 

Second, the process ajr^roach to teac^iing science was emphasize 

in one of the object iy^ of the Science for the Seventies program. 

/ - J* *» 

Followinft/the broadcasting of SFTS oriented 
televis^ lessons, children in participating 
clasaifooms Will exhibit a measurable significant 
in(;^ase in their facility in>he us^ ofi sciencje 
processes. (SFTS and Instructional jTeleVisiorr, ' 
Phase I, project Report, 1973, p. 211>. 

Ihirjd, since the major medium of SFTS wais instructional tele- 
itf LaS a natural course of action that the ins t^rtiment used 
luate the ability of -SFTS stydentis' to (jiemonstrate their 
tanding of science processes be; presented over te Levifiion. 

/ . /'/ ' , ' 

Thus, early in the ptoject it was a^pided thati 

T"/ 

A telei^ised test will reveal increased student' . - 
competency in the usie of science processes andJ ^ . 
attainment of aims end objectives of SFTS (SFTS ^ 
Phase I, Final Report, X975, p. 24). ^ ^ 

i ' ^ : ■ \ 

The? test, entitled the Televisipn Test of Science Processes 
(TTSP>, ha9 been deyeloped by_^vid R. Torrence f?:om the Test of 
Sc^.ence Processes by Robert Sher tannenbaum. Tannebaum's original 



test has ninetyrsix Items and was developed in 1968. It is admini- 
steted in a multi-media format., A 35mm color slide stimulus is shown 
which contains a realistic representation of objects, a table of data, 
or a graph. In the test booklet, the visual is reproduced in black- 
and-white above the appropriate multiple-choice question. Students 

respond to the .interrelated visual displays, and questiotn and then 

" , •# ■ . 

select^ what they believe to be the correct answer. ^ 

Torrence (X974) had .sfeveral tests^ to choose from but decided on 

Tajinenbaum's teat because it offered several advantages. 

Of the available tests purported tfo mfeaffure - 
science processes, fhe Test p£ Science Processes 
-of Ur. ^Tannenbaum was chosen because it was an • 
instrument whose statistical data showed it to ^ 
be a reliable instrument possessing face or 
content validity. Al«o, the processes categorized 
as subsets by the author were bcjth general and 
comprehensive. Most} applicable to the SFTS 
demands was Its visufai presejitation mode wl>ich 



had implication for 
('p.'2)\ 



the television format. 



The Television Test of Science P::oc<jsses as developed by ^ 
Torrence is a forty item miflt iple-cholce test administered in two ^ ^ 
half-hour sessions. > v ^ 

' I In^the first"^ half-hour, istudents watching the videotaped program 
• . . ' . ' .J ■ 

axe/'introduced to ithe test , , Ji)nstructed .on how to ^f ill out ishe 

.answer sheet, and then anpweS: questions 1 through 20. For each 

question there, is a visual diisplay, the television narrator asks* the 

question, and the student chooses the correct iinswer of the apprap- 

riate tiiultiple choice "question in the test 'booklet , marking Tiis' \y 

ariswer- on ,the '^pswep iheet. The yisual display is left on the screlen 



for an exposure time varyir/g from 16 to 58 seconds which was. aeter- 
mined by Torrence in a' te^t-item time study . JThe, sfecond half -hour, 
covering questions 21 through 40, is conductecj in the same^ fashion. ' 
Appendix A contains the questions and information found in the test 
booklet. In addition,. ToriAence also developed a Teacher's Manual 
which familiarises te^chersWith the test and the concepts underlying 
it and gives them ins true tions/on how to proctor the^ test. Aiipendix" 
B contains the .Teacher's Manual. 

By the spring of 1975, Torrenc^ had completed the following' steps 
in the development of the TTSP: one, the question corit^nt of 
Tahnenbaum's test had been subjected to a panel of recognized experts 
on elementary science tip determine ^ the content validity of the test^ 
as ^pt>licable to intqrm^iate leve!. elementary students; two, 

by th<s Experts had beeji rewritkien so' as 




quest3.ons deemed appiof 

to have a Readability ad^quate'^^ elementary students of th^^ above 
lev^l; three, a pilot tejL^vision t jst and the associated prirtt^p^terialSi 
had been producedrfourj J pilot stu^y had been performed to ascertain 
the appropriate te^t-item exposure 'time; five, the videotape had been 
edited to coniform t^o the time Hata; six,, a seco^uj pilot study had 
been- conducted which e.xposed the^ffesC 1:b a larger number of -fftudents^ 

gather dat-a' for. item analysis; and seven,, statisi«d;cal parametl^rs 
had been established which enabled the item anglysis/to provide a 
refinement of the questions -c^oroprising .the final /form of the te^t 
and the video production had been edited accordingly. 

le is/^to this <poii«> that, a broad exposure of the test to as 
large a nmnber of students as^ possihle is n6<!fessary, so that norms , 



for target populations can be ascertained. The test is scheduled to , 
be broadcast in September of 1975 over WPSX-TV, a public television 
station associated with The Pennsylvania State University. The test 
will te broadcast during the day as part of the in-school service 
program. ° ^ 

Need for the Study , - • 

* * 

This broadcast of the Television Test of Science Processes 

at • - 

over a public television ^ tat ion in Sept^emb^r of 1975 will represent 

. * . ' * , " I* 

an innovative effort in education. While educators have used 

♦ *- 

telfevisibn as a medium for testing before » as will he discussed in 
Chapter/ II; no instance is revealed in the literature in which such^ 
a carefully developed achievement test has been publicly broadcast as 
part of *an In-scKool service program to students over a, large 
geographic area.' Thus, if successful, it will set a precedent for 
both future administrations of the TTSP and the development an^ admini 

stratipn of other television, tests. ^ > 

^ ^ , • » ' • ' , tj 

The success of this broadcast of the TTSP depends upon close 
cooperation of a large number of professionals in edueiation. It is 
anticipated that classroom teachers, eiementiaryo school science 
chairpersqnB, elementary school principals, sthool^^superintende^s, 
district designates of the educational broadcast council, central 
management of J the broadcast ^Qouhcil, and members of the SFTS project 
that had developed and produced' the test will all have the^otentiaj. 
to make an Important contributioVflto administration of /he. test: 
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Considering then both Dh£.Jjmov4?tl^ nature of the upcoming 
broadcast of the test and the amount of Coordination necessary 
between the large number of thosfe directly Involved, It has been 
recognized that there would be a 'definite nded for a system of 
procedures and guidelines to Insure an adiQl^lstratlon of the TTSP 
free from the problems usually encountered In such a multi-level 
cooperative effort. 

Statement of the Problem 

The problem then to which this study will address itself is 
posed in the follQwln^\ question: Can a model of administrative 

procedures and guidelines be developed which when implemented wHl 

- '» ■ * 

res^t In the successful administration of the Television Test of 

Science Processes? 

Significance of the Study ^ 

the development and implementation of this model, if it , result^ 
in a successful administration of the TTSP, will be significant ^f or 
three "reasons: first, if this prototype is valicjat^d, it will set 
an administrative precedent for bioth future jbrpadcasts of (the TTSP 
and the development and adtainistration of otW television test» of 
this type;' second, it will provide Torrence with the data •needed to 
derive rtdgrms for target .populations; and third, th6 students 
participating in this administration of the 'test can be retested 
at a later date for comparison purposes, ' , 



Definition of Terms' « 

The following terms are defined so as to provl4e the reader with 
their precise meaning In the context of thl^s siudy. 

Achieve ment test — An ^achievement test is an Instrument 

f . • ; 

designed to measure Informatloti acquired by an Individual from a* 

• • ' f ' . 

program or programs of Instruction, * ' / * , , 

Broadcast television — Broadcast television Is the unrestricted 
transmission over open air of - a television 'signal which can be , 
rec'elved by any standard television monitor. It Is contraSted^wlth 
closed-circuit television on which reception of this, signal Is limited 
toV monitors directly Connected to the transmitting agency by * 
microwave or cable link, * . . • . 

Instnictlonal television — Instructional teleylslbn la a 
television program or a series of television programs, th^ purpose 
of which is to educate or instruct rather than entertain. 

Television test — A television test is a , visual test presented 

— . , ^ < 

"by use of television as a delivery medium. 

Test administration via television ~ Test "bdministration via 
television Is-frhe administration of a verbal 'test ^sinsL television 
to explain test procedures and, in some. cases, actually present the 
printed tesf- questions. 

Verbal test — A verbal test is aiv, Instrument of evaluation 

which assesses primarily by means of information presented in a 

* ■ ' . • ■> 

verbal or print format. ^ 



Visual! t sst — 4 visual test Is ati Instrument of evaluation which 



assesses ^ 
pictorial 

Sommaty 



I 



Imirlly by means of Information presented In a visual Jor 



As Inistrikdtlonal television has beccjjpeT' firmly eatrenched In the . 

" • " i ■ T, ^ ^ - . ' '/-^ 

Amertcafi educational s'truQture^ Its uses have expanded greatly. This 
study will coitcentrate on one* of .theses expansions, namely test 
admlnlstratloijj using broadcast television as a medium. si)eclf Ically , 

J V 

coi^cemed with the development and Implementation of a 
^dW^nlSterlng the Television Test of Science Processes fo 
a large population of elementary sWpol students geographically 
dispersed throughout central Pennsylvania. ' ^ * 



It will be 
model for 
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<:HAPtER II 

REVIEW OF THE LITERATURE . . ' ' ' 

Method ' ! . , ■ ^ ' ' . • -r-" " ' • 

'-Due to the 'innovative nature of television -testing, it was felt 
that research on the topic would be. sparse and scattered throughout' 

« 

- the numerous journals and publications in the fie]«(i of education. • • 
With thife in mind, a computer search of both Psychological Abstracts - 
* and ERIC was performed using the descriptor? television and tes,ting. 
While this method produced' inany "hits" not directly related to the * : 
topic of television used a? a 'testing medium, the folJ.owing studlea 
proved to be of relevance, they can.fre grouped into three basic 
categories^- namely, visual testing, test- administration via television 
and testing by television, ' . V 




Visual Testing - . _ "^V^ / 

Gross (1969) argues persuasively ^ more testtng. a 
bv visual mediaT She points out that in- receftit years there has^ 
been more^nd more emphasis on student learning by visual presentation 
but tests have remained locked in print. - She concludes that .it is^-^^ 
unfair to the student to test verbally what her has learned visually. 

The verbal testing of visually presented, material . 

is often frustrating to' the studenjt and is not a' - , 

true measure of what 'he actually knows. In this 

way, this verbal testing in the presence of visual 

presentation stimuli is akin t0 the older IQ tests 

which are not a measure of gentifral Intelligence 

but largely a measure of white middle-class 

vocabulary standards. (p. 35)..- 
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The vavis of research In Instructional film and television In the 
-1950' 8 brought a pioneer attempt In visual te^stlpg. Carpenter, 
Greenhlll and others (1954) developed a motion picture test for the 
Track Vel\lcle Repairman Course at the U,^ S. Amry Ordnance School. It 
prese^nted 'a series of situations which, shoved a malf unctix^n, an 
liicorrect mechanical procedure, or a parts Identification or * , 
relationship problem. After, piloting *the test with 316 students, 
high reliability was^demonstrated and^^eV concluded t^^at: ^ • 



Film tests can be practical to admluister , • 
^ objectively scpred, and- maVe it possible to 

' • ' test areas of performance not amendable to paperV 
• ' pencil testing. <p. 52;L): ^ * • / 

■ ^ y ; ■ ^ ■ ^ ' ^ ' ^ :V 

Another 'use to- which film tests have^ been put is in psychbmetrics 

(Selbert and Snow, 1965; Selbert, 1970).' Among the psychological 

• •- ' . • * * ^ 

film tests developed are visual tests of long and short term memory 

• »■ " ^ ■ . ■ " 

and various tests, of the^ principles' underlying visual perception. 

Curtis and Kropp (1961 1962) conducted an extensive series of 
exploratory research^ in visual testing Uslngv slides. ^ intelligence, 
achievement, vocational interest', and adjustment tests were all 

presented in the visual projection mode. Variables 'tested Included 

^ \ , ' ■ ■ ^' ■ ' ^ . . - 

intrafitem exposure interval, inter-item exposure lnteirval» Items 

per exposure, audio-visual dnd visual item presentation, and item 

difficulty. arrangement. These studies were done with an eye toward 

television testing, but the cost factor prevented any research in 

/that medium. 

' Among their more important conclusions was that this type "of 

■■ . / *■ ■ 

visual testing was more "feasible for the administration of tests 
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of cognitive abilities" than for test of attitudes such as vocational' 

interest. (1961, p. 79h They also noted that : 

The projected mode of t^st administration" can 

yield a greater coverage of test* content in a , ^' 

unit of time than can the normal ifiode of pest 

presentation. It would seem that for togni- 

tive tests i . test-taking time can be reduced 

without sacrifice of mean, variance/ relia-r ^ 

bility or vaUdifcy. (1961; p. 79). * 

Another interesting finding concerned the value of the students 

*' . ' , . « 

seeing all the items on a, test. in a once-only type of exposure. - 

The projected mode of^ admlnistration^inci;eases - ^ 
the'number of items which Ss attempt. Altho.ugh^* 
^ this would be expected on the b^sis of exposing - 

all ijtems to Ss., there seems to be an additional 
advantage In that the procedure weakens the ;^ . 
'cautiousness' respotise set of some Ss^ perhaps ^ ' ■ ^ 
because S£ know that an item will be presented >r 
one,, and only one, time and it prevents persevering ^ 
' on difficult items which appear early in th% tjpst. 
(1961, p. 80). ^ 

Stoker, KropjJ, and Bagshaw (1968) reported no sighif icant- 
diffefence between '^a booklet version and a projected slide -version ^ 
of the Occijpational Interest Inventory- Test. In an interesting , 
reversal %0f this type of research, Hpffer (1972) found t^at the ^ 
ability to trouble &\iO£>t television receivers could he as effectively 
. t^ested by using photographs of tTie screen as it could be actually 
using the television itself./ 'm' ^ - 
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Test Administtation Via Television 

Th§ first uses to which television was put , in. education was to 
simply transmit lectures from a=-«^l^sropm or studio. Similarly, t>ie 
first uses of "television in the realm of educational testing was 
simply as a dellvety mediiira, * ; . ^ 



Woodward (1964) and Pens^nger (1969) descrll^^e ope Such^^stem^ 
which had beeK^ In use at San Josf State College since 1959. Wheh 
standardl^zed tests, such as the Graduate. Recofd Exam, were to be ' • 
admlnlstere^y^only one; person kxiowledgeai)le In xhe pxocedures of the 
test was required. This pers'bn would descrj.be vhow the te^jt would be" 
conducted^ .tell how ''to fill out the adiswei: "sheet and give any other 
instructions from the xampus television studio. These Ins truetHx^ns 
would be^roadcast £ntd several clasisrooms contali\livg approximately 
thirty students and a proctor. The 'classrooms were Ifnked to^the 
studio by a telephone llne^ While the students were taking thcr^xam, 
'a clock showing ^how much time was remaining would be broadcast over, 
the system. * * . %^ 

Woodward strongly*" emphasizes the advantage of such a system. , 

' • ■ ». 

* In conclusion"^ television testing offers a very 
, . : ' real solu^pLon to the problems of testing large 
/ groups with «^tandaj^zed ^xaoiiiBation^« Problems^ ' 

associated with larjge-group testing are virtually 
eliminated/ Fewer directions are misinterpreted 

students; complete ^tandardi;^tioh is 
achieved; a smaller additional - |)rb£esslonal staff 
^Is required for testing. (>964, p-. ISSji,: 

* Diamond (1968) in his' teview -of instructional television also 

Strongly "favors this method. He points .^ut that: , , 

- By utilizing aft enlarged te^^ing form^ student 
errors in filling out the required information 
can be substantially reduced , at a savings of 
hundreds of ma^-*hours required for hand scoring.- ^ 
In addition, comple&e standardizatiofl is obtain-^ * 
^ - able at the University o]^f Miariiii, the f irst t ime ^ 
this^approach was applied,' student error in 
completing the required information — student 
seat, and section -number — was reduced by 
80 percent. -^(1^68, p. 383)., \ . 



Hgpkins; Lefever,vand Hopkins (1967) compared adminiatratton of 

. * ^ . .'.-^ , 

the Metropolitan Science Test by either closed-circuit television or 
>■ " ■ ' ' , ^ ' ■ 

teacher. "These subjects were 3,375^ fifth and sixth grade students 

•in twenty elementary" schools. ' , 

The results showed no overall significant difference in means for 

either type of 'administration. However, it is of interest to this ' 

study that those students in; large classes (approximately 70 students) 

scored ^significantly better with TV administration of the test. 

* '*c . , * ■ " " ' - 

A logicaLji>ctensien of this use of television for testing is 
repoi^ted by, Damth:o\ (W72) in her reyAew of presenting psychology 
courses over ■^"sed-circuit television at the Univers^.ty of Akron. 
She describes- a system by which the printed questions, themselves, 

were broadcast. t ^ ' v 

Multiple choice items are typed oh 5"x8" grefen 
note cards u§ing an IBM Selectric typet^riter with r ; 

drator type. Questions are caref^^y edited to ^ 
fit a 3"3c4" margin on the note card. In a tele- 
vision test, thBse testvitems on the cards are 
projected over the TU, receivers. Each item iTs ' ^ 

shown for 4tf seconds and read by the TV lecturer. 
At the end of the test each item is visually shown . 
again fpr 20 seconds.' This second projection gives 
the student ^n ^opportunity to check or change; his • ^ 

^ , answers.' Test questions are not read by the 

TV lecturer oft the second projection. Then the 
answer sheets^ are collected and the test items 
projected for the thirf time with the correct 
. \ answers provided. This third projection gives 

.^X^ immediate feedback '^nd reinforcement to the / ^ ' \J 

dt^idents. ^In a 50-minute period, a total of 

' 30 laultiple-choice items are proj«<?ted three 
ti;iies.x^(1972, pp. 186-187). \ 

■ • . ,1. ' - \ r 

Dambrot reported; that, ixCitially,' studjents in the eourse reacted 

' ' ■ : ■ .'^ "... 

quite negatively to this typfe of test. After some modifications. 



among them reducing the number* of test Items and improving the quality 
of the graphics-^d videg, the student attitude improved to the point 
where 76 percent of the students were satisfied with this metho^ of 

testing. . "^^^^^v^,.:^ . ^ ' 

' Two studies have shown THe classroom teacher to be a Significantly 
better test administrator. Munger/(1972) had tlfe Gates Reading Survey 
test administered over closed-cirdui^ television and by^cl^ssroora 
teachers. She found higfter reading scores^ for the student^ in the , 
latter group'. Hegstrom and Phill,ips (1973) used both a television 
narraJtW and classroom teachers to administer two '^specially prepared 
tests of reading assessment. ^ They also' found a significant difference 
in favor of» the claseroom teacher. This suggests the possibility that 
tests of" reading ability do not affapt well to administration over 
television.'^ / , V 

Testing "by Television ; . - 

* " Tes^ting television is a hybrid of visual testing and test 

administration via television. The medium is used hot only as. a 

delivery mejchanism but also as an integrikX part of ^he testing process. 

Whi'le jaot truly academic tests, two rei^tiyely wi^ll-known early 

. examples ol^ this type \of testipg were the Nati^^l Drives Test 

- - \ > . 

presented by CBS-TV In t^ie middle 1960 's and the l^tlonalUlitizenshtp 

Test. The format was tW^cal for tests of this genre — a^hort 

scene or demonstration was ^hown and then questions .were asked 

concerning it. Alper and Lei^^(1969) foutid that persons who^-^ad ^ 



viewed the National "Citizenship Test performed significan^y better ; 

/ 

than non-^viewers on a t;rue-f alse questionnaire cqyering the infprma^iion 
presented on the test both one month and six months after the test/. 

Fargo » et al., (1967) administered the Peabody Picture Vocabulary 
Test (PPVT) by television. The parallels between the PPVT and/the 
Television Test of Science Processes are striking. Both a(re aehieve- 
ment tests for elementary school children which began as visual tests 
administered individually by projected slides andL then were adapted 
for presentation over televislbn. Therefore, it is particularly 
interestitig to note that when the PPVT was administered to third and 
fifth grade children at the University of Hawaii Elementary School, 
no significant difference was'fpund between an administration to a , 
group by television and a oneron-one'teacher to pupil administration. 

The researchers also pointed out that the TV test had several 

advantages. It was more economical in timd and pei^sonnel (2 1/2 man- 

/. . ■ ■ <« „. *- 

hours. per student using TV, 25 man-hours using individual 

f ■ ■ ' ■ 

administration). TV could make the test more uniform, and thus more 

consistent and reliable, by using one narrator/test administrator, 

and also by repeating the same videotape" at different times and in 

different schools. Alsp Classes would ^t be interrupted for 

^ individual administrations. - ' 

Two other studies have compared a television test with the 

- . ■ 

former mode of administration. Graham (1971) describes how, a dfag- 

■ , \ 

nostic test in electronics making extensive use of schematics an^ 



diagrams was coiiverted f^rom a f ive-screen slide presentation (one . 

rt • * ' - •« 

each for the stem and four alternatives) to a television test. No 
significant' difference in scores was;found between th§@e methods of 

presentation. As the author ppints out> the five-screen slide 

■*•*>»» 

version had the advantage ^f being able to display the stem and 

alternates simultaneously, but the television test was much easier 

♦ , , ■ » . ' 

to distribute and ad^nister. 

Stallings (1972)" compared televifio^ and audltf' presentations of 
the MLA French Listening Examination. Once again; there was no 
significant difference in test scored. The advantage here would be 
with the television test as a source of non-verbal cues- from the 
speaker. ^ 1 " 

- ' Another use o^. television testing of second-language skills is 
described by Thomas (1975). It is a purely visual test. Tl>e 
television screen is divid)^ into quadrants, each presenting a 
siigKtly different drawing. T^iese drawings ^ are Wso repfoduced in 
the test booklet. A sentence is\read\ln the\ second language and 
the student places a checkmark in the \corneE of the picture that 
depicts the sentence besk. / \ 

A bit of informal research was carried -out during the administration 

■ \ ^ - • . 

ofv this test when there was a *pow^r loss in sevW^l, classrooms and the 

' . . ^ ■ \ . • ' 

sentences had to be read orally bj|- the test proctors. Analysis of these 

test scores showed tRere was no significant difference between these 

and the classrooms t^at had received the TV test. ' 



Taggart ktid rtoore (1972) report Qn an Interesting use of tele- 
vision testing In the language la^ ora^ofyii Students, watched a serle§' 
oi videotaped situations played^ on a jtelevlslon monitor', heard tlie 
language being spoken over headset^, and then responded orally to 
questions 6n the scene uaing a cassette recorder. At the end of the 
test, the audio cassettes were collected and graded.y Each test wjLs 
30 minutes long and contained three slt4iatlons and a total Of 50 

questions. ' ' . 

^ \ ■ ' , 't ' > ■ 

The situational type testing popularized by CBS has also been 
used by other educators. Project NuTEx (Landls, Masonls, and toye, 
1971; Masonls, 1971) attempted to Improve the Natldnal Teacher 
Examftiatlons by creating t^o televisiqin tests. The first was ,a 

multiple-choice verbal test. ^It was found to be statistically 

* ' - • - 

invalid. • 

The second test displayed various classroom situations and asked 
questions on each. Unfortunately, while the construction of this 
test was covered extensively, no report was made on whether gr not 
it was piloted. . * 

Examples of situational testing catl^be found* in indiistW^l 
training and medical education. Ford Motor Company. (Deimel,« 1975; 
Sundt, 1975) makes use of* it in their sales training. videocassettes. 
A situation involving a salesman and a customer is displayed and then 
the viewers are quizzed on the appropriateness of the salesman's 
actions. . These .programs- are reported as having been quite well . 
received at the dealerships. ^ ^ >^ 
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The; lietwork for Continuing Medical Educafiop (NCME), supported 
1^ a drug company, distributes vldeocassettes to doctors for a fee. 
.These bl-weekly 'programs provide updates on medlcsa^ Inforioatlon. y One 
method they use Is Informational and situational ''self-assessment" 
quizzes; in each videotape to Increi^ey'physlolan involvemek^^^ 
(Jacobson, 1975) • The NCME has als| pr^duc^d the ''National Antlbl^ 
Therapy Test" and distributed it to 370 hospitals and medical schools. 
For each question on the test a clinical -situation is presented and 
then a choice , of four drugs is given. Ther^ is a pause for the doctor 
viewing the vif.deotape to make a response^ t>en a pdneFof experts 
discuss fhe pros and cons to e^ch answel^:^ P4ucatlonal and Ingustriaj: 
Television (i976) reports that over 30,000 physicians have t^fcen the 
test in the past two years,. q 



/ Summary v . , / , 

Moat importantly a review of th^ above studies shows ^at 
televisibn testing has great potential for educational evaluation. 
When* constructed with care it is ^s effective as normal classroom 
administration of verbal testing and it offers sev-efal advantages 
implicit in the medi\im.^ ^ ^ 

FirLt, It^can expand. the quality and scope of current test ^ 
capabilitll?! bv/addinip^ elements to the test. The need for 

visual testing is becbmiiig more apparent as our culture' becomes ^ 
more vlsua^iy oriented. In addition, television testing can present 
situational e^^ments which make the test a much more sensitive 
instrument to assessing the judgmental abilities of Its audien^. 



«' Second, a television test recorded on videotape is inherently » 

■ '■■ • ' ' r V,. ' ' : 

a standardizeid test; 'It c^an t)aye repeated exposure with no variation 
in items, presentation method, and pacing* ^ ' ' 

Finally, a televisidn test can be extremely cost effective as 

several of the studies have demonstrated. There can bei a considerable 

■ ^ - * - ; ^ ' • ' , . 

decrease in the man-hours required for administering and procjtoring 

the test. As a result, k test can be exposed to a larger student/ 
populatioxi with the same number of professional staff. 

The studies "did not offer much insight into the development of . . 
a procedural model' for the administration of the Televisiori Test of 
Science Processes^ however, siMlply because no study- had developed 'any 
such procedures which could be applied 'to the .upcoming broadcast, in 
addition, most of the studies had been conducted With populations oj ^ 
under 500 subjects, while the TTSP had the, promise of a much; larger 
group.- Thus, it was apparent, that the model developed and dmplemented' 
for the TTSP would, if the results were successful, have decided valufe 
for . future- administrations of the TTSP and other tests administered 
over broadcast television. 



» 



. ' CHAPTER III ' • , 

• • ■ * '1 

PROCEDURES AND DESIGN 

; ' ;■, / ' ■ r 

Introduction 

The problem of thls-^slpudy as defined in Chapter I, was to 
deslgiv a procedural model for th,e adminlsl:ratJ.ott of a visual test 
over broadcast television w^ich, when implemented ^ would .result in 
^Pjr:^ a successful administration of 'the Television Test of Science 

Processes. The TTSP was scheduled for J>ro^ca6t over ,WPSX-TV, a 
public television station in September of 1975.^ Since a review of . 
the literature produced no' information or guide\ipes on the ,admini- ^ 
stration of this type of test, it was realized that the model would 
have to be developed from a detailed analysis of the problem and 
woul^have implications not only for this f irst'broadcast of the TTSP, 
but also for future administrations of this test and other television 



tests. What follows then is the analysis of the problem, the 

development of the model, and the design for the implementation of 

* ' - 

the model, to the upcoming broadcast of the TTSP. 

i 

Analysis of the Problem 

The TTSP is a standardized achievement test which has been 
scheduled to be administered over broadcast television to a 
potentially large population of intermediate level elementary school 
students in September of 1975 to assess their knowledge of science 
processes. The first step in the analysis o^f the problem .is to 
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divide it into sub-problems which are identified by posing the 

following questions! 

1. How can the elementary schools within the broadcast 
- - range of the television station be informed of the 

upcoming broadfast and encouraged to participate? 

2. 'how can those elementary schools that desire to 

participate be identified? 

3. How can the test materials, that is, the test 
booklets, answer sheets, and teacher's manuals, be 
distributed to the appropriate schools? • - 

4. What methods and procedures would be necessary to' 
insure that the individual classroom teachers are 
knowledgeable in thp proper manner of filling out the 
answer sheets and other proctoring information? ^ 

5. How can the test materials be returned to the 
administering agency? * 

6. How can the answer sheets be scored and the results 
interpreted for arid distributed to the participants?^^ 

7. What methods can be develbped to evaluate the 
procedural model developed for all of the above? 

' Taking these que^ions into consideration, th? next step in the 
analysis is to build a viable framework to seek solutions t6 the 
sub-problems they represent. An examination of the issues raised by 
these sub-problems yields three distinct areas of interest. 

^he first area that musfc be considered is the identification of 
those institutions which will take part in the administration of tfi^ 



■ ✓ ' 23 - 

test. Those whom the test administrators think, would be Interested 
In participating have to be Informed of the upcbmlng event, they 
have to be motivated so they will want to take part, and those which 
Indicate a willingness^ to participate have to be Identified. 

The second area Is a matter of logistics. The test and associated 
materials have to be distributed to the participating Institutions, ^4.^ 
speclaj. 'Instructions have to be given to those iservlng as classroofn 
proctors and administrators of the test, the test Is then administered, 
and all test materials have to be collected and returned tf> ^he 
administering agency. 

The last area Is concerned with evaluation. The tests must t>e 
scored and the results Interpreted. These results with the appropriate 
Interpretations must be distributed to those who took part. Finally, 

the admlnlstratlotl of the test Itself has to be evaluated to Identify. 

. •• • # 

procedural flaws or poor administration practices that must be 
corrected. > 

By Isolating t^ various sub-problems Into one of the three areas 
of Identification, logistics, or evaluation, it noW becomes possibly * 
to begin the development of the model of procedures for the admini- 
stration of a visual test over broadcast television. 

Development^ of the Model , 

The model ^developed to admlhister the TTSP la depicted In Figure j 
1. Its purpose is to provide the outline of a syste^p of procedures 
that, if adhered to, will result in the successful administration of 



stage' I: IDENTIFICATION OF TEST POPULATION 



Function 1: 

Publicize Test 



Function 2: " . 

Solicit interested Institutions ^ 



Function 3: 

X Identify Participating Institutions 



STAG^ II: IMPLEMENTATION OF LOGISTIC^ SYSTEM 

4- / 



" ^ / 

Function A: , 

Implement Materials pistrlbutlon System 



Function 5: ^ 

Verify Proctorlng Procedures 



Function 6: 

Present Test 



Function 7r \ 

Collect Test Materials 



STAGE III: "^VALUATION OF TESTIIJG PROCEDURES 



Junction 8: _ 

Acquire T§st Results 



Function 9: ' 

Distribute Test Results with InterpretatdLons 



Function 10: 

Evaluate Test Administration 



Figure 1: Model for the* Administration of a Visual Test of 
Achievement Using Broadcast Television as a Delivery Medium 
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a.yl8i]ial tes^ using broadcast television as a delivery medium. Both 

its stages and functions correspond closely to the areas identified 

/ ' ... , 

in the analysis of the problem. - Each stage will now be diBcussed* in: 

ligh.t of the particular requirements of the TTSP* ^ 
The first stage, identification of the test populatibn, would 
be critical to the success of the project^^ Essentially, the elemen- 
tary school administrators and teachei^s are going to be approached 
with a test they have not been expo^d td before and, in adclition, 
a test which -is to be presented by broadcast television. Due to 
these innovative aspects, the information publicizing^ the upcoming 
broadcast of the test and the methods used ^ invite the 
participation of the various elementary schools will have to be 
designed for maximum effectiveness. This will j'esult^.in a test ^ 
population large eqough to derive norms foi^ target populations as 

required by >Tbrrence . ^ ^ . ' 

* . \ 

Design Sot Implementation of the Model tjo the Administration of .the 
TTSP 

The next step In the pi^ocedural design is to take the Bod^ 
derived from the analysis of the problem and apply it to the devc 
lopment of specific procedures for the administration of the TTSEi 
The specific proceidures derived from e.ach stage aild function of the 
model will now be examined. ^ . , 
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^ STA6E I: IDENTIFICATION OF TEST PPPUUTION 



1f 
-» 



1/ • '1 

Functfon 


1: 


publicize TeBt 

: — i 1 , • ' — 








-1* — - 

function 


2: 


Solicit Interested Institutions 




Function 


3^ 


. Identify Participatingllnstit^tions^ 



Figure 2: The Model, ^Stage I 



Stagje 'l! Identification of Test Populatioa , ! . 

During his pilot research described In Chapter I, Torrence ha^^ 
established readability of the TTSP at a fourth-grade level. On this 
basis, it-^has been decided that the projected test population should 



be^fif th-grade students! Since it would be early in the schools year. 



they^hould have the reading skills adequate for the comprehension of 
test questions. . i 

What needd to be done, then, is to pdblicize jthe test to the 
elementafjr sqhobls within the broadcast range of WPSX-TV, Invite and 
encourage thfelr "participation, and provide a means for Identification 
of-»'thos6 willing to participate. 

About seventy ^rceat d£ those schools ate members of the 
Allegheny Educational Bro^dc^sT^CctofUii (AEBC). The AEBC is an 
instructional television school service""a|&icy which has as its main 
function tfhe acqtrisition and broadcast of i^jistructional television 
programs for classroom lise. Tl\ere would be several advantages in 
working cooperatively with AEBC.dJuring the implementation of Stage I, 



First, as mentioned above, the majority of schools In the WPSX-TV 
viewing area' are members. Thus, by working through the agency, a" 
slgn;Lf leant number of the area schools can be reached. 

Second, WPSX-TV has bfeen allotting time for the broadcast of 
programs acquired by AEBC. Thus, AEBC has a working reilatlonshlp iJlth 
both thq^ station broadcasting the TTSP and the schools that are \ . 
potential partflclpants. ^ ' . • 

/Third, AEBC has been presenting th^ "Science- for the Seventies" 
television lessons. Thus, the member schools are already acquainted 
with the parent project 'and, as a result, ^would p^qbably be receptive 
to the TTSP. 

, Finally, e^ch member school district or school alppolnts-an AEBC 
District Designate, us|iially a teiacher. The designate functions ^as a 
llason between AEBC and the school district or School. Thus, this" 
person could serve.aa a topr^nator for the TT^ at the school level.. 

With all these factors taken into consideration, is 
concluded tiiat the^lmplementatlon of the model can b^ greatljj ^ 

facilitated by working with AEBC. With tttls in mind,- the ftollowln^ 

^ ^ * . " " . . . ■ - ^ ■ ■ ■ / 

procedures are specified. ' . . • 

Function 1: Publicize Test. The EiieGOtive Secretary of AEBC 

should contact each member schooK district or 

school by letter. This letter will publicize 

' ^ the test by giving tbe dates and times that the 

test will be broadcast . Another channel is 

' Ithe AEBC Utilization Conference which will ,|)ei 



' / held on August 26. Since most District • 

• ' Designates will be in atX:endance, a presentation 

should be made- to -further puWipize ;th,e,test. "^In " 
addition, tfiis presentation can r^terkte and. 
reinforce the information contained in the intro- 
ductory letter and provide the designates with 
the opportunity to see 4>ortion|p of the test and 
^ ' ask any questions 

\ > Function 2: Solicit In^ter^sted Institutions/ On^re again-, the 
* ^ letter from the Executive Secretary and the 

presenta|:ion a^ the Utilization Conference can 
solicit patticl^atlon by explaining the back- , 
► . ground and rationale n&T^e test^ and by 

delineating the benefits of taking part. ' *\ 
Function 3: Identify Participating. InstltutidnsV* The letter 
• from the Execjitive Secretary can also 'idtentify 
^ those willing^to ^participate by giving f 

> " : • " ■ .. i ' ^ ' - ; . 

- . instructions ion how t^o. obtain test materials 

by a cut-off-date. \ . 

• ■ : . " . ' ' ■ . ^ \ ^ ' 

\ ■ . . 5 * ' ^' - ,A . ' ' . . . 

^Stage II; Implem'entatlon of Logistical System -^--^^ 

* A metjhodlcal system has to be created to distribute the test 

booklets, . answer sheets, and teacher's manuals to the participating 

schools identified in Stage' I. The classroom teachers, have to be 

informed or all proctojing procedures, and, after the test has been 



STAGE' II : I>ffiLiEMENTATION OF LOGISTICAL SYiSTEli 



Function. 4: 


Implement Matetlals Distribution System 

— ^ — \ \ 






Function 5: 


— ' ^ ^— : \ 

Verify Froctorlng Procedures 




Function 6: 


Present Test \ 




Functlop 7\ 

\ 


\ ' ■ 
Collect Test Materials .\ , 




Figure 3*: The.ModU, StSge 11 v 
t^ea^all test matetlals have to be retuitned. In keeping with the 



principles of^lmpllcity and efficiency underlying* the development 



.of this stage^ trhe~XollfiSi4a^^ havd bSen devised. ^ 

Function 4: ^ Implemetit Materials Distribution System. WhenJ ' 

the' schools registered with A^BC to take the TTSP, 
they could at the same time request the approximate 
number of printed test materials that ^they reqikre. 
Once the cut-off-date Is reached, registration. for 
the test will be plosed, a master l£#t 9omplled, 
and ,the printed materials will be mailed to the* 
appropriate school districts which will* have the 

ponslblllty for distribution to the individual ' 
schools.. If an indlv;tdual^school is participating, 
the]r would receive the mater ia?!8^d4Tect^yr^ 'Contact 



\ 
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by tel^ephone .will be made to Insure that the 
materlalsv have been received . 
Funct^lon 5: Verify Proctorlng Procedures. General details ^oft 
taking the t6st\ and filling gut the answer sheet 
, " ^ are covered during the opening moments of Part I 

of the test atid In ^he Teacher's Test Manual 
developed by Torrence\ (Appendix B) , which will be 
among the materials mailed. However, more specific 
' * Instructions will be necessary. The Students? 

\ ■ ■ ' ■ ' x ■ ' 

. - \ * names have to be penciled In on the answer sneet 

so It can be returned to them for Part II of the 
test and the name of the school di;strlct has :o beV 
placed there also In case an|r/pf the answer sheets 
Ire mixed up before they can 

additional Instructions can l>e given during th^ 
telephone check of materials received. 

■5* p . 

Function 6; Present Test. Bj^cause some schools wlsh^TnR to 

participate might not be able to tak^ the test « ' 
on the scheduled broadcast dates. It should be 
acceptable. If they .videotape the test for admlai- 
stratlon at a later time. Only twcr conditions^ shoul4 



be sdored. These 




X 



be Imposed — they will have to pf:Oj^lde thl^ videotape 
on their own equipment and they' will have to 
administer the test and. return ^^he other test 
materials by a cut-off date; It i^^-mitlcipated 
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that this option should not yield any Important , 
dlffererices between those whc^ chose to ^vl^ieotape 
for a later playback^ ^and those taking the test 
• when broadcasts * 

N- ' ■ ■ . ■ ■ '■' 

Function 7: Collect Test Materials. The printed test materials 
can be mailed back to AEBC after the test's 

^' ' • 

administration. The answer sheets should be 
separated from .the vOther matearlals for scoring. 



STAGE III: EVALUATION OF TESTING\PROCEDURES' 
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" FuncMon 8,: Acquire Test Results 






\Functlon f: Dlis tribute Test Results with Interl)retatlons 


1 \ — ■ 1, 


I ■ ■ 1 


Fun^ t ion 10 : Evaluia t e Tfest Admin 


tratlon 



Figure 4: The- Model, Stage III 



Stage\lII;\ Evaluatloa of Testing Procddjures 

Tl\e thitd strage of the model presents som^ easy decisions and 
some dlf^lcuXt otti^s.. Th\B difficulties exist because the return of ^ | 
test sdvortes to\ the participating Ihs'tXtut^ons will be a sensitive; \ 



matter atid\must\ be handled ^delicately. \The tTSP has never been exposed 
*tQ a group of the size that S^s anticipated^, .norms haye hot been , 
established; dnd, thus, the o^ortunities for . misinterpretation o.f the 
resqlts ti^ay be great. It would be unfortunate If /further 'development - 
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of tKe TTSP is hampered by a misunderstanding of test results by one 

or mote participants. ' These factors 'fiave led to the adopt iwi of the 

following procedures. ^ 

Function 81 Acquire Test Results. The answer sheets will be 
coded and scored by Examination Serviceso^ 
University Division, of Instructional Services, The 

' ■ ■ ^ , Pennsylvania Stat^ University . \ The SFTS t>roject 

has used this. service in the past » with Satisfaction. 
Function 9: Distribute Te^lt Results with Interpretations. For 
purposes of this- pilot large-scale l/roadcast, the 
only results sought will be the/byerall mean f or " > 



the entire test poipulation and t|vS mean for each 
. school district on school. If aiy school district 
\ 6r school wishes to break down t^e results any \ / 
further tih^y will be supplied with- their own ansi^er 
sheets and 1 test tey, so"* that tjhey will be able t^ 

determii^e individual results if Ifehey wish. These 

^ . ■ ■ * ■ 

results -^^ overall mean and schoolXdistrlct mean ~ 

will be made known only to each Individual ^ , 

participating school district or school, that is, 

\ nd other participant. would be told what^another 

^school district or school has scored. Also, all 

test results will be provided with- the appropr^ia^^ 

interpretatiojis. Misinterpreted results Vould be 

worse^than no results at aLLl. / ' . 



Function 10: Evaluate Test Administration. To/assess the 1 

. ' , / .1 ■ 

Impact and value of this f^irst lirge-scale* 

•■ . . • 

^ administration of /the TTSP, a follow-up question- 

naire will be designed and mailed ouf^ after the 

test has been given^to the classroom, teaohefs 



participating :jLn the aclminis'tratipn. It will 

;ive res^hSes -'o'f 



not only indicate the af 



the teachers^ but also prol(;^^^daba on'^suck 
Information as size ofMc^Laq^e^i, whether the tele- 
ion recdi\^e^||||^ 4i6)|pr oi* mono^rome, tl}e 



vis 



quality- of the receptlort and on. 



The results ^ 



ll br 



combined with the , 



of tills questionnaire sha 
impressions of the project planners and others ' 
-to evaluate the broj^ect^s successes and failtrx^^s. 




Summary , 

A review of the literature pr{o|lirced no useable 
problem addressed by the study, namely the deve lopment 



lelines^for the 



'procedural 



model for th^ administration ^Jbf^ a vifeuatntest oyer broadcast television 
and the implementation of this model to the admitflstration of the 
Television Test of Science Processes. this chapter, then, an 
analysis of the problem, was performed; from this analysis a mcrdel . 
was developed, and this model^was applied to the TTSP which resulted j 
in the development of specific procedures for .implementation. 



CHAPTER IV 



^ re^iJltsVf the 'Implemen^tation oj the MOBEL 



'A 



In£fbductlon " , ^ . 

^ The jproblem Investigated by this s^tudy was tl\e development of a 
procedural ipodel for the administration of a visual test over 
broadcast television and the Implemelitatlon of this model for the 
administration of. the TelevjLslon Test tff Sclenecr Processes*""^ * , 
r^sview of the literature produced no administrative guldeliMs\|.f 
•val|Liei so an analysis df the problem was undertaken and from st a 
model* was developed. This model was then applied to the upcoming 
broadcast" *^of the TTSP and specific procedures were derived frpm it. 
This chaptet will describe in detaj^l the implementation of the- 
procedures and what occurred as a result/ using the stages and 

functions of the inojdel as an outline for discussion. 

t ■ - \ • / 

Stage' t: , Identification of Test Population 

Function 1: Publici2e Test. 

Publicity for the test was fifst generated by a letter. This/was 
written by Mr. Will\E^m BarnJiUfrt of AEBC» dated August 14, 1975, and 

addressed to superintendents of member^ school districts and district 

Y- , y ■ - ■ > 

designate^ (Appenddx C). In addition, a presentation was made to the 

district delegates during the AEBC Utilizat^oo Xon^erence at University 

Park, Pennsylvania* on August 26. Both of these activities were also 

utilized for Function 2. . * , 
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Function 2: Solicit Interested Institutions 

Barnhart's letter also solicited participation by explaining the 
procedures of the test, the reasons behind it. and how each school 
would benefit from taking part. He referred any questions that 
scjhool officials might have to the upcoming AEBC Utilization 
Conference. 

The presentation at this conference on the TTSP encouraged 
participation in the following manner. The information contained in 
the Barnhart letter was summarized, the opening moments from Part I 
of the TTSP were shown along with the test booklet and teacher's 
manual^^nd questions from the floor^ were answered. One concern 
expressed as whether it would makd a diff^ence if the test wjere . 
viewed' in color dr monochrome. It was explained that the test and 
the visuals for it had been designed with thris in mind and that it, ^ 
shouldn't make any difference^ Other questions also concerned tt^e 
development of the TTSP itself. Some' delegates wanted figures on 
reliability and' validity which were unavailable at^that time. Others 
expressed' jthe opinion that the TTSP was at a reading level above fifth 
gr^de. With the latter in mind, some of the delegates wanted to 
know, why tlie classroom teachers couldn't read the alternative answers 
to their students during the test. 

All questions were answered faking use of available^nf ormation. 
While >j3ome delegates seemed to be dissatisfied with the replies they 
had received, , the ^ majority expressed interest in the upcoming event 
and several delegates confd.rmed that their school districts woulcl 
participate in the test. 



The final method used to solicit Interested Instructions was 
peri^onal cot^tagt, either by staff meiobers of AEBC or tfie SFTS 
Fr^fect. In several Instances this proved to be a most Influential 
. factor In' an Institution's decision to take part. 

Function 3: Identify Participating Institutions 

Institutions evidenced their desire to participate by requesting 
test materials. Instructions on how to do this were Include^ In the 
Bamhart letter 'ami reiterated' at the conference presentation. 

Between late August and September 8, which wap established as 

the cut-off date for requfst, fourteen school districts Indicated 

that they would participate In thjs test. An additional two public 

schools and three pargchlal schools also indicated their willingness 

f 

.to participate. Table 1 contains a list of the school districts and 
'Individual schools that took part in the TTSP project, the» number of 
tests requested and' the number of tests scored. Figure 5 sHows the 
geographical distribution of the school districts and individual 
schools throughcryL central Pennsylvania* 

Stage lit Implementation of Logistical System 

Function 4: Implement Materials Distribution gystem 

The various school districts or individual schopls requested 

materials by contacting AEBC either by letter or telephone. These ^ 

requests were forwarded to a staff member of the SFTS Project wKo 

then compiled a master list for distribution of materials which were 

I 
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then mailed. This proved to be a source of difficulties, especially 
if the requests were made informally or by telephone. In three 
Instances, due to delays in either deceiving or forwarding these 
requests, the test materials would not have been received in time if 
jnailed, attd^ad to be brought to the appropriate schools by car. 

Teacher *s^fenual8 were distributed as requested, or if no 
specific number was supplied, in the ratio of one manual for every 
twenty test booklets and answer' sheets. Each. package sent out to the 
schools, thea» coatalned the nudber of t^st booklets and andwer sheets 
that had been requested and the appropriate number of teacher *s manuals 

Function 5: Verify ^roctorlng Procedures 

A telephone check was made shortly before the broadcast of the 
TTSP with each school district or individual school to Insure that 
the test materials had been received. In addition, special ins true-' 
tlons were given and any questions about the pr6cterltig process were 
answered. 

These /telephone calls proved to be troublesome and time- 
consuming. El^mentairy dchool teachers and administrators can be very 
difficult to reach by telephone *durlng the school day and leaving a 
message does not Insure that the proper information will be received. 

Function 6: Present Test - ' 

The test Itself was broadcast twice during the third week in 
'September according to the following schedule: 
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TAB|(E 1 

SCHOOL DISTRICTS AND SCHOOLS PARTICIPATING IN THE TTS? 



Name and 
Location 



Number of Number of 

Answer Sheets Aqswer Sheets 
Requested Scored 



Altoona^Area School District 
Altoona, PA 16603 ' 

Bald Eagle Area School District 
Wlngate, PA 16880 

Bellefonte Area School District 
Bellefonte^ PA 15823 

Berlin Brothersvalley Area School District 
Berlin, PA 15S30 

Brookvllle Area School District 
Brookvllle, PA 13823 

Clarlon-Llmestone Area School District 



900 



266 



280 



81 



Strattanvllle, PA 1625^^' 



Curwensvllle Area Schlool District 
Curwensvllle, PA 16833 . 

», 

Indiana Area School District 
Indiana, PA 15701 

Lock Haven Cathplic School 
Lock Haven, PA 177A5 
« 

NortTiern Bedford Area School District ' 
Hopewell, PA 16650 

Punxsutawney Area Scliool District 
Punxsutawney , PA 15767 

St. Francis School 

_ CljEBrfield,' PA ° 16830 

St. Peter's School' . - 

Somerset, PA 15501 

Shade Central .City Schools 
Cairnbtook, PA 1592 A 




115 



150 , 



22 



29 



85 



750 
245 
233 

« 

190 

82 
138 
318^^ 

21 
lOA 
133 

21 

25 
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TABLE 1 (contMj 



Name^nd 
Location 



Number of 
Answer Sheets 
Request e4 



^ Number of 
Answer Sheets 
Scored 



Spring Cove Elementary School 
Roaring Spring, PA 16673 

Union Area School District 
Sllgo, PA 16255 ' 

Warren County School District 
Warren, PA 16365 

Westmont Hilltop Area School District 
Johnstown, PA 15905 

W4J.llamsport Area School District 
* ^ Wllllamsport,-PA 17701 



155 



100 



280 



240 



149 
82 



650.. 450 



172 



209 



Part I ^- 9:30 a.m. , ^Septeifiber 16 . 
Part II-- 9:30 a.m. .September 17 

Part I 2:00 p.m. , September 18 
Partli— 2:00 p.m., September* 19 . , 

Approximately three-quarters of the participants took the test on 

these dates. The remalnfng quarter videotaped thes te^t and took it 

at a . later date with the understanding that all test materials had 

to be returned to AEBC for scoring by November 15. ^ . 



Function 7: Collect Test Materials 

Each school district or individual school was responsible fpr' 
mailing the completed .answer sheets, test booklets, and teacher's 



manuals back' to AEBG. ''All of the test- materials .dlstrlbute4 were 
received by the cut-off date.* " * 

s i' ^ * ' ' ' * ' ' ' ' ■ * ' 

Stage III: Evaluafelog^ of Testing Procedures 

Function 8: Acquire Test Resu'lts t • ' * , 

Of^ t^he approxli^ately ^200 answer sheets distributed, over 3500 
were completed ,by Vtudents and irec^ived ba^k- foe sbofing. Approxi- 
^mately 50 had to be discarded because the student h^d not bi^n able 
to* tdke one part' of the test due to absence. ,Some schools/had not 

received enough' answer sheets, so they had used phot ocopies^^br 

(? ' • 
several. groups of students. As a result, It was necessary to ^ 

... ^ » ■ ' - ; - . ' ' ' 

transfer each student's answers- to ah^drlglnal answer sheet. 

The answer «sheeftis ti(ere. sotted and then J>r^ought to Examination . 

Services, University Division of Instruct ibn^t Services, at The 

' * , 1. p« ' * • 

Pennelylvanla State University fbr coding and scoring. As planned,' 

■ ' ■ • < . ' ■ \ 

an overall mean and a me£ln for each school district or school was^ 
obtained. The total number qf answer sheets scored was 3,479. 

Function 9: Distribute Test Resul^ts with Interpretations 

SlncB^Mrr David Torrence had developed the TTSP, it was decided 
that he should have the major role ip distributing the test scores * . ' 
with the accompanying interpretations. He sent to each school 
district or individual school a letter r^uesting further information 
which would aid him in determining target population nqrms (Appendix D) 
In this letter, he also explained his responsibility in making the test 



\ 



results and the appropriate' Interpretations available to them. 
Mter receiving their replies* and establishing the necessary 

norms, he reported to each Individual participant the apj>rpprlate 

' ' 'I * ' 

scores with Interpretations. » \ 

Function 10: Evaluate test Admlnlstratldti v ^ 

Approximately two weeks after the broadcast of the TTSP, copies 
of the "Teacher^s Survey", which contained 13 multiple choice 'questions 
were sent out to j:hose. serving as^test coordinators In each school 
district^ or school for distribution to. the classroom teachers. The 
numbeY of questionnaires sent put was determined by the number Qf , 
teacher's manuals which had been requested*' With each was a <y)ver 
letter explaining the purpose of the survey and a stamped self- 
addressed envelope. 

' ' ■■ • ^ ' ' ... y \ 

In total, 165'sjirveys were sent. Of these approximately' 20 wei:e„ ^ 
returned blank because the^ number o-f classroom tBachers^ho had 
participated had been overestimated. The total number of surveys 
returned fof tabulating" was 115 which Is aljpost 80 percent of th^ 
adjusted total. This high rate, of return reflects quite positively 
on the strbng^ Interest the participating classroom teachers had for 
the TTSP*. 'This Interest was also reflected In the written comments - 
which had been requested at the end of the purvey.. , v - 

Tabled contains the "Teacher's Survey" and^the' tabulati^fl 

)l^inns > 



results, ffo the left of the actual questionnaire are two coli 
of figures. The digits In ^the ^flrst column marked "N" are the number 



oi^xresponses made for the choice located to the right of It on the / 
same line. Xhe second column represents the Individual numbers asy 
a .percentage of th^ entire, resj^onse to the question. In addition 
to these figures ba^ed on^the multiple choice responses* many 

Revealing and worthwhile comments were also received.^^ An analysis, 

•» . . * < 

of both yields several insights into the project. 

'First ^lid most important was the problem bf television reception 
which was not conducive -to good performance on the test. Note the 
responses to Questions four and six. Nineteeh percent of those 
responding considered the/^^i^eo reception inadequate and 30 
percent indicate'd that ^ecWrfical difficulties were present throughout 
the test. The disturbing conalpsion is that one child djn five' took 
the test in substandard condi£l)ons. The written responses also 



ref lect^this. These problem/s occurred during. both the open-air 
liroadcast and the yideptaped playbacks and were blamed on either 
poor reception or incorr/ct* sizing the scteen on the television 



monitor. The following ^comments reflect thl« 

- , The receixtioti of WPSX-TV at our Sichool is very 
poor. Thl^s is both audio and visual. Several 
times during both sessions the picture and- . - ^ 
sound would fade t)ut for approximately -five or 

/ ^ mpre seconds... If it had not been for the poor 

^ reception, 1 would welcome having the test 
televised again. . . 

' . ■ ^ . ■ ' : ' * / .. 

The *t^st will not be a fair evaluation of my class 

i becai^se of the terrible technical trouble we ~- 
experienced. . .The television set we must use lis a 
' 19 inch pottable» Jil^k and white and the rece^Xion 
was- very ^poor. . . 
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Television ^tape was of poor ai^dio-^vldeo quality. • • 

Sometimes complete picture ^ould not be seen over 
the TV screen... ^ ^ v ' 

Some oF the -picture question^ had the most Important 
parts cut of f th^ ^bottom such as the scale of' miles 
..^ *' when they too use It for measuring. * 

Other teachers Indicated that there were too many students for 

one monitor and those In the back could not distinguish relevant 

aspects'^of the yldeo^^or that glare was a pr:oblem. The. average class > 

size according t\>^ the questionnaire was^ between 15 and 30^ In <a group 

of this size, Xo make the acute visual discrimination necessary* there 

should be t;wo monitors. ' ^ ' , . * 

' ' r - ^ ^ v; . ,^ • . ^ . . ^ 
The results of ' the survey, demonstrated that the' distribution of 

test materials had been successful In that o^er 90 percent of the * ' 

teachers received the Teacher 'fe ^est Manual . This figure may have been 

■ . , " ■ " ' . " " ' ■ . ' ' ■ ./ ■ .. 

even bjjLgher, but there seems to have been some confusion as to l<iftv^c 

the manual was as Indicated by one teacher In a written comments / 

I am not sure that -I had the use of a manual, 
f I believe the test had a sheet of directions..*/ ( 

The Infort^dtlo.n contained In the manual was generally Well/- 
received," On^e again, the survey> Indicates that only three percent 
foi^nd the m^ual*s orientations to the test Inadequate, while 94 percent 
found the information it contained laasy to uhderstand. 

An Important conc^n^wlth the teachers wa-s the level of compfe- 
hfenklon of ^the test. The majority of those responding (61 percent) 
felt that the^TTSP was to€^ "advanced for fifth graders, especially 
those students with below-average reading skills. ; y ^ 



The&e are some of tTieir comments. ' 

. seemed to'be ti^.adv^nced for oui? f if th\ grade 
students. Many said th^y guessed in answe^irinjg., \ ^ 

' . i would like to have my class take the^test at ^ ' ^ . 

the end of their 5th grade year instead of the ^ 
beglnnlitg of the year. A lot of the material ^"^'^ • , 
presented on the test will not covered untll^ 
^ the end of the 5th grade year./, ' * , • . 

Pupils of low level were having difficulty^ With ' 
Coordination of looking at the TV and listei\^ing, ' 
then reading bookl^tis ahd markihg their answer. » 
This type of test, t'felt, was' suited for one level. ^ ' 

The teachers also indicated that motivation was a piroblem for the 

student and in at least a few instances for the teachers, , themselves. 

the responses to Question 11 show that ^,68 percent of the students, as 

judged by their teacher^, reacted indifferently or unfavorably to the 

test.' This may be due to the issues discussed ^bovev. pob.r reception 

and perceived test difficulty, but th^re seems to be two bther' factors 

t>whiQh also enter into it as expressed by this teacher. 

: ' / ■ . ■ 

^\ % * In my opinion the test would »*have more validity... 

(a) do not tell the students the test is non- 
graded. I found seve'ral students responding without' 

> giving thought «or coftsideratlon , to th6 questions. * 

(b) more time for teacher preparation. In other 

words, jperhaps the day before testing. ' ^ ^ 

Other teachers also mention these problems. One comment, in 
pairticular, was' most revealing. . ' i • \ 

» I was not satisfied with the reason we gave the 

test. It has ho introduction, litjt.le explanation, 
and seemingly no relevance for us...A9 I .tecall , 
instructions to the teacher — little h^lp was 

♦ given to motivate or ^x^lain the test. ' . 



Despite the reservatl^ons expressed abpv^J, the overall response 
to the TTSP by the teachirs was favorable. In Questi^^ 12 of the 



survey, they were asked ta^ assign a letter grade to their overall 

* a 

♦ 

experience with the TTSP. More than three-quarters of the teachera 
gave a passing grade, with 42 ^rcent giving either an **A" or a "B". 
Also lndlcat4^ of their positive response to the experience wjia 
that slightly less than three-quarters wanted their students to have 

the opportunity to take the test again, with 35 percent especially 

* <\ 

favoring the mode of television presentation. ' - 

o 

Summary 

Theoconcem of this study has been .the development of a procedural 
model for the administration of a visual test over broadcast tele- 
vision and the implementation of this model to the administration of 
the Television Test of Science Procesi^s. Since a review of tlie ' 
literature yielded no useful guideline^, the problem was analyzed. 
This analysis produced a developmental model, which was then applied 
to the specifics of the TTSP. This resulted in a schema of 
procedures for each of the ten functions of the model. These 

procedures were then implemented, as discussed in^ this chapter, 

• • ' . . . 

resulting in the administration of the TTSP to approximately 3500 
fifth-grade students in fourteen school districts and five individual 
schools throughout central Pennsylvania, du^ng the months of 
September and October of 1975. " . 
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CHAPTER V. • 

SUMMARY, FINDINGS, CONCLUSfONS, AND RECOMMENDATIONS 

Statement^ of the Problem 

This study was donoerned with an expansion of the'' applications 
of television to modern education. Specifically,, it dealt with the' * 
use of broadcast television as a delivery medium for a visual test 
of achievement. The problem of the study was how a model for a 
administering such a^^st over broadcast television could be developed 
and implemented. 

Summary 

The tegt that was the impetus for the study is the Television 
Test of Science Processes, a visual test of achievensent for elementary 
school students. It was developed by David Torrence for "Science for 
the Seventies,^' a series of instructional units for elementary school- 
level science, combining television demonstrations with integrated 
classroom .activities. The TTSP bad reached the point in its 
development wh^te a large-scale exposure was needed to establish 
norms for target populations. • As a result, the test was scheduled 
for broadcast over a public television station, as part of art in-school 
service- program, in September of 1975. ' ^ 

Due to the innovative nature of this problem, a search of the 
literature was undertaken to determine if a previous sttidy had 
established any guidelines which could be adapted for the TTSP « 
administration. The literature reviewed produced several studies in 
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each of the three related areas of visual testing, test administration 
via television » and television testing. FtQiP these studies It was 
concluded that television testing could offer several advantages. 
Some of these were >fisuallzatlon» standardization, and cost- 
effectiveness. None of the studies dealt with testing situations of 
the scope of the upcoming broadcast of' the TTSP, however, and 
therefore had not developed any applicable guidelines. 

As a result, an analysis of the problem was undertaken. It was 
first divided Into sub-problems for consideration. This resulted In 
the development of a model consisting of the three stages of Identl^- 
cation of participants, distribution of test materials, and evaluation. 
The functions were Identified for these stages. Taking each of those 
functions Into consideration, specif Ic ^sets of procedures were 
developed for the administration of the TTSP. These procedures were 
then Implemented. • " • ^ 

'•-V ' ' " - ..' 

Presentation and Discussion of the Findings 

The application of 'the specific procedures derived from the model 
resulted In the successful administration of the" TTSP. Approximately 
3500 students took the test and three-quar&ers of these took the 

^ H * . 

te^t on the dates that it was broadcast. The remainder took Xhe 
test by videotape at a later date. All procedures derived from the ^ 
model worked acceptably; however, some difficulties were encountered. 
Each function of ,the model will now b6 reviewed in light of the 
experiences gained from this administration of the TTSP. 
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Stage I: Identification of Tdst Population 

Function 1: Publicize Test and Function 2: Solicit Interested 

Inetitutions. Time limitations resulted in the combining of these 

fufictions. Both the ^.etter written by the Executive Secretary of 

AEBC to the area school districts and the presentation to the district 

delegates at the AEBC Utilization Conference resulted in an ample test 

<> * ' * 

population; however, a wider publicity cai^paign would have probably 

.A" 

resulted in an even larger, audiencje. > 

Some problems were encountered at t*he conference presentation in 

■» 

that the presenter did not have sufficient knowledge of the develop- 
mental aspects of the TTSP^nd, thus, was unable to answer questions * 
of this nature satisfactorily. It is suggested that future presenta- 
tions of this type should include a person qualified to handle these 
questions. This would ejiable those teachers and administrators at the 
elementary school level to be more aware of the purpose' and rationale^ 
of the test and, thus, be more fully supportive toward its upcoming 
administration. 

Function 3^ Identify Participating^ Instittitidns. For ^he 
administration of the TTS^, institutions that were going to 
participate were identifiied when they requested test materials. Some 
difficulties were encountered as' a result of this and will be 
discussed in the review of the next function. 

Stage II: Implemeiitation of Logistical System ^ 

Function 4: Implement Materials Distribution System.."^ Since no 
"fqrmal system was established for requesting participation in the 
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broadcast ahd test materials, most of the requests were received on 

Informal basis by AEBC and then forwarded to a member df the SETS 
project who mailed the test materials. This informality and the 

forwarding through' several people resulted In the delays mentioned 

, • * ' ' • » ' ■ 

In Chapter IV. Because of this ,J thij^ee improveio^iits are recopmiended, 

/ ' * i 

F£rst,,more time should be allowed for the schools tOv respond 

- *: ■ A 

to the Invitation to participate In^ thej TTSP. Since they had a 
time span of less than one month to re^^pond after the first letter^ 

compounded with the. staff limlt,ati:ons cf these schools d^^rlng the 

* * - * ■ ■ . . . ' 

.summeV months^ most requests were received ^Ither shortly befpre the 
closing date for requests or shortly after It. This did not facilitate 
distribution; but» fortunately, allvrecuests were accommodated. 



Second, requests: for TtSP tes^t rd^ier±§fia should be made on some 
sort of a standard |orm/ This form woi>^ indicate date of request, 
the number of^^^t booklets, 'answer sheets and teacher's manuals'^ 
required, and thdi^correct address to whi|c1^ they should be^ sent, "^his 
would reduce great'ly, if not eliminate, il^isinforfhation due tto 
informality ^of^ requests, especially ^hoset bV t;elephone. ' 

. Fiq.£(lly, these formalized requests styould be returned to the 
party responsible for the distribution of tef t .materials and should not 
be forwad^ed through any other agency. Th|is|should eliminate a^ 
delays between the receptloj^ 6f requests ^nd Ithe mailing of materials. 

Another problem encountered in t:he nvaterials distribution system 
was that several schools did not receive; enou];h answer sheets. Extra 
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care should be taken and, If necessary, sinaj.1 over^estlmattlons 
should be made to insure that this does not happen because the 
/transfer of answers from a photocopy to an original answer sheet Is 
a time-consuming process and Introduces the possibility of errors In 
the^^propess which would reduce the student's score. 
^ Function 5: Verify Prbctorlng Procedures. A telephone call 
to give special instructions Is not 8uffl<^lent. These Instructions 
should be typewritten , duplicated, and attached to the Teacher* a j 
Manual. ^ A telephone check shqrtly befoire the test is of value, ' 
however, to insure that the test materials have been received and 
to answer lany questions. ' 

The post-test suryeiy indicated that two more improvements could 
be made in this area. F'ltst^ more care must be taken to Insure that 
ehe teachers receive the test manual in time to read it thoroughly. 
It cannot be assumed ^that this will happen just by mailing the 
manuals to the district or school AfBC representative. This person 
must know that the^ mater.lals are to be distributed Immediately and 



folldw-up checks Should, be made. * ^ 

JThe second problem area lies in the' fact' that some of the' 
teachers "felt left out, in that they did not underatand the "why" 
behind t^he TTSP. The test needs more than the paragraph or -two 
intrcrducing it in the current test manual. The early sections of 
the manual should be revised and expanded to a fuller explanation 
of the backgroi^ri^ of the TTSP, why and how it was developed, what 
the' TTSP evaluates, the benefits of the testing experience, and how 
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It was developed, what the TTSP evaluates, the benefits of the 
testing experience » and how teachers and students can derive the 

most from this experience. A section could also be added on 

' . ' ■ * *. . 

mot;lvatlng the students to p^erfbrm at a maximum ovua non-graded test< 
In addition, there may be a need for some fate-to-face meetings 
between teacher^ and the personnel behind the TTSP to provide the 
extra relnfoi^cemenl: of personal contact. The above steps are needed 
to produce ^ teacher who* Is enthusiastic about the prospect o£ helping 

to admlnl&tet the TTSP in the classrlpom- and whose attltade toward' 

• ' * \ ' 

the test c^n do nothing less than to motivate students toward a 
maximum performance. ' ^ - / 

Function ^6: Present Test. .This is entirely under the conftol 
oT the broadcasting station and the classroom teachers.- Since the ^ 
survey revealed that too many students viewed the test In unacceptable 
conditions, efforts should be made in future test administrations to 
make the teachers responsibJ.e for reporting and eliminating any 
reception problen|s befoj:e the test is broadcast, or at worst , before 
the actual test c^ue^tions are presented. . . ^ 

' Function 7: Collect Test Materials; Few -difficulties in this 
function were experienced. Nearly all the participating school • ' 
districts and indivirfuarl schools were diligent and prompt in the return 
of materials. Si5^, however, ^ere unsure of the procedures to be 
followed, even though they had received ^ telephone call giving these 
special instructions. Once again,, this argues for the distribution of 
all proctdring procedures, including special instructions, in print form. 
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Stage III: Evaluation of Testing Procedures 

• . ' ■ ' ■ ■ /■ •■ 

Function 8: Acquire Test Results.. The only difficulties 
encountered in this function^ resulted f rom ,the use of photpcopied 
answer sheets wl)ich had to be recopied. Other than that, it^'vas 
simply a process of sorting the "answer sheets and bringing them to 
Examination^ SeWices of The Pennsylvaniar State University which 
performed the coding ^and scoring. 

Function 9: Distribute Test Results with Interpretations. As 
mentioned in Chapter IV, David. Torrence performed this function after 
gathering information on each school in" regard to location, groupln)^ 

of classes, and the type of science program in use. With this infor- 

\ 

maitionvand his extensive knowledge of the TTSP, he was able to 

..provide the participants with the individual results and appropriate 

interpretati-ons.^ " -i 

Functit)n IQ: Evaluate Te§t Administration. This was performed 

quite adequately by developing ^ survey questionnaire and diistributing 

it to the teachers and administrators who participated in tl^L 

experience.. The provisions that were made for written resp'<6nses 

proved to be very valuable, since many of these comment s/were ^ 

/ * 

particularly revealing. This feedback functioned as a/source pf ^ 

% / \ ^ 

constru(ffive ideas for improving the administrative pjrocesses. 

Conclusions 

1. The first'' large-scale administration of thfe Television ' 
Test of Science Processes was a success. /Approximately. 
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,3500 fifth-grade students* in schools Chroughoiitf 
central Pennsylvania were evaluated by the tes^. 
This provided Torrence with the ^necessary da^ 
. for the establishment of, norma for target populations. 
Since this administration of th^ TTSP, was sucQs^ul, 
the^tocedural model' that was^ developed and imple- 
mented for this purpose, which was the problem of this 



^ ^Upl^udy, h^s been validated. 

. ' 3. Slncife the worth of the model has been deii(iojistrated, 

it should be implemented with the suggested' revisions 

■ f in. the future administrations af the TTSP. ; V 

4^ The model can also' be considered as ^a prot6type that ^ 

^ is adaptable to thp administra4;dtons of other tests 

, ■ ■ -[ . ^ ■ - . . 

When broadcast television is usdtd as a delivery medium* 

' .« '"t 

«* • 

"Guidelines for Implementing the Model ^ ' 

Since .the worth of the model has been "demonstrated in the 
successful administratJfDn* of the^ Television Test of Science Processeis 
over broajjcast television, it is hoped^that others in the fields of 
indtructionalv technology and education will find it useful in future 
administrations/of visual tests over broadcast television. -This holds 
not only if or future administrations of the TTSP, biit also for other 
visual tests yet to be developed. 

The f irst implementation jDf the model, which Has ^een the 
.subject o^ this study, has provided many experiences that have given ^ 
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mu6h insight into {he administrative procedures Involved. With 

^ ' . A, ' ^, , 

thesd i^xperlences In mind, the followihg guidelines are offered for 
the future Implementation of the model. Once ag^in, the model* 
itaelf, will be used as%a. jframevork for the discMssion. 

/ ^. — - ... 

Stage I: Identification of Test PopuiLat Ion ^ \^ ^ 

Function 1: Publicise Test. Every resource ^vailable may be 
used to generate publicity for the u|coming broadcaAt^of the' test. 
Among the possibilities are notices in newspapers anci, educational' 

publications, promotional spots oh educational radio and televisions 

, I ■ > 

especially on the station that will be broadcasting the test, and 
presentations by the developers of the test at Vconf erei:\ces, workshops 
and schools that are potential participants. V « ' 

Besij^es disseminating general Information, such aa^ the nature^ 
of the test and the date and time it will be broadcast; it is also 
important that detemined efforts be mj^de tp ins.ure that potential 
'participants are also aw^re of the purpoSe and Taflonale of the 
test,, so* that they may b^ mbre,,,supp9rt^ it^iipcoming 
administration* • ; . , . , 

Function ;2: Solicit Interes4:ed Institutions. AH. forms of 
publicity generated in Function 1 caa clo$e with an' invitation to 
write br call the test fityministrators for further information. fhi;3 



should be the m|in ^diirce of potential pa^icipants^ ^ 

/. ■ ■ ■ , ' . S " i ^ ' 

. , conjunction with the above, thpse schools that the test ^; 
adminisl^tr^tors believe would be the most likely prospects should be 



'cDttCactad-^omally- by iett« contact 
shbuld also^ iDadle ain tp provide the flndl impetusf for . 

}: J 3: Iden-fW^^^^^ ^rtst^utlons. Iderftifying 

, fete that Will participate should be done formally, especially - 



'since requests for the necessary t6st materials is done at 
the^aw(i timi^i, ^Sonrces of confusion xati be elimliiated by providing ^ 



intere/ted in^^^ a standardized requj^st- form when they' are 

solicitied by letter^ ThiS^ reque^t^ form would noX only represent a t 
commitment by the Schools returning it,, 'b,ut also s»^uld contain an 
area for 'indicating the needed amoiii^t ot test booklets, answer sheets, 
and teachers manuals. In additioo, a closing date foX requests ^hould 
be established- that aljLots adequate/t^me. for 'both the return of < the 

■■■■ ■■- ' " . • 

request form and the distribution of the printed ibaterials. All 
reqiiests should be returned to the party, responsible for materi^s , 
distribution. ^ - ; \ 

Stiage II: Implemer\tatlon of Logistical System " ' 

^ Functimr^ Implement Materials Distribution System. |lequests 

for the printed test mate^rials should be processed as.sooti as possible 
.after they are received. Any special ins'ttuctions* for ,the upcoi^Lng 

administration of the test should* be printed andl included^ with the 

teachers manual. , ' ' ^ . 

Function 5: Verify Proctoring Procedures,. ^ .^Approxima^ly two 

- ' _ . ' ' ' '''^ \ i 

weeks befpre the broadcast of the test^ a telephohe checl^ should be 





nisure that the printed 



made with each participating pchool to 

.rixiatei^als have been received and to ans'di^er any questions about 
proctorlng procedures. If ^circumstances and resources^..peT?mlt , 
there coyld be much benefit derived from a workshop held in each,// 

' ^ •■ • ■ ' ' ' . ■^ . • • . ' ' ■ 

school by the' test administrators for the classroom tei^chers that"^^ 
will be participating. This personal contact should insure thak * ^ 
these teaghers understand "their role in th^ administration Qf the 
test and would provide ah opportunity to encourage their enthusiasm 
and ^constructive criticism. ' ' - " 

Function 6: Present Test. Responsibility for •this' function 
rests with the broadcasting station and the classroom teachers 
proctorlng the test. It is vital to the success of the test admini- 
stration that any problems o^^^elevision xeceptlon in the classroom of 
the participating schools^oe identified and elimlnate<| before the 
broadcast. Classroom teachers must be liiade awarje of their role in 
reporting any difficulties iii' this area and school administrators ^ 
should insure that these difficulties are resolved. 

Fun^ion 7: Collect Test Materials. In the printed special 
Instructions distributed with the teachers manuals, there should be 
a description of the procedures to be followed in labeling and 
returning the completed answer sheets and the other printed items., ' 
In most circilmstances, they can be mailed back to the party respon- 
sible for their distribution. As part of these :^structions, a^ 
closing date should be provided for the return of these materials and 
their prompt return should be encouraged. * . 



Stage lil: Evaluation of TestlngTProc^dures 

Functfott 8: Acquire Test Results. In the case of a latge-I ' 
$cali^ administration » sucb as In this s£udy/ provisions should be 
made In advance, for the machine scoring of the answer sheets. With 
the nedessary arrailgemients made> the test results can 4}e acquired 
qulikly and easil) 

Function 9: Diktributa Test Results with Interpretations.^ 
Participating school sh<^ld be provided with the spotes of their own' 
students that tpok the te^ with the appropriate Interpretations.* The 
distribution of these Interpreted tesj: results* should be done with 
discretion and confidentiality. .It cannojt be overemphasized^ that the 
test participants tsu&t understand what the , results mean arid hoW these 
resuit^^can'be properly appll^^ , > 

Function 10: Evaluate" Test Administration; ^ A wrltteri sl^ey, - 
similar to the one, developed for this administration of the Television 
Test of Science Processes, Is a worthy tool 'for'. the assessment of \ 
the overall administration. The high return^*rate of the TTSP 

.questipninalre demonstrates the positive Response toward this device 

* " ■ i' » ' ^ ■' . , ' ' "* • " ' 

by teachers and administrators alike. The multiple-choice questions 
can provide revealing statistical data and, by encouraging the 
additional written responses, further insight Is gained into the 
classroom admlnisttation pf the test. The results' of this survey can 
preside the necessary feedback for tire improvement of both the tesjt 
ind the processes involved^ in its administration. . t 
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^Recommendations for Further Study * ^ 

Qased on the results and the conclusions of this study, ^he 
following areas for further study and consideration are suggested 
^ 1. Turther Implementations of the model should bts' 

performed In the' administration of visual tests over 
. broadcast televl^lotl. This Vlll serve to further 

\. ^ * . ■ . ^ > ' w 

vallc|ate and Improve the model. In particular^ the * 
. « ^ model should be Implemented id tb€r future admlnlstra- 
V tlons of t;he Television Test of Stlence Processes, 

sliice it was developed for this purpose. 
' 2« The television Test of Science Processes should t>e 
y readministered td the student population who wene / ' ^ 
evaluated by it durfng* the coUrseN)fi*-dtiis stydy. 
This would provide valuable comparative data both on 
students performance and on the recommended suggestions , 
for the moclel*s "refinement. ' » . * 
.Research needs^ to be done on what at^ the'^be^t methajcTs 
to Imprbve the\af fective responses ojf the classroom 

^ - : - / A • ■ 

teachers who proctor the TTSP. Preliminary efforts 
should iriVolve the revision and e^pans4,on of the 
Teacher's^ Manual and, if possible, the imple;nentation 
jDf In-tservlce workshops for these teachers in the schools 
that are p^^rticipating^ in Ifutur'e administrations.. 




Much study needs to :be done on what Is necessary to 
optimize the reception of television in clasisrooms. 

\ • 

A9 this study. has pointed out, too often the classroom 
television monitor is the limiting factor in thla 

* reception* The causes of poor classroom reception^ be 
th&y human» mechanical » or electronic , need to be 
identified and solutions need to be proposed^ imple- 
mented, and evaluated in order to correct this • ^ 
lingering problem. 

Whether or not there is a significant difference in 

* taking the TTSP in color or monochrome needs to be 

investigated. Question 3 of the survey showed that 90 

percent ^of the classroom television monitors were 

monochrome, while the test was Recorded in cblor. There 

may be no difference^ since the test was designed with 

the knowledge that some groups would be viewing it, in * 

» ^ It* 

monochrome, but the size of this group demands that 

statistical tests be made to insure that there is no 

\ ■ , 

si-gnificant difference in the group means. 
Further research aridp development should be done in the 
general field of, visual testing, and in particular ,* in 
the /iBe of broadcast television as ia delivery means 
for this type of test. The results of this" study have 
demonstrated that this technique is a promising tool of 
educational evaluat;ion. \ ^ ' . 
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DO NOT TURN THE PAGE UNTIL THE TEACHEK TkLlS TmJC DO SO 

TELEVISION TEST 
SCIENCE PROCESSES 

INTRODUCl-iON . 

% 

This test Id different than many of the tesjCs you take. It viU. involve 
what you see on the televlislons^een. Choose your answers f rom' among 
the choices given in this booklet.* Mark your choice on the spi^cial ; ^ * 
answer sheet. ^ / ' * • 

Below are tw> practice questions/. These will help you to understand ' 
what you are to do. Wait for the television introduction before you >* - ; 
begin. i • 



1Q9. Here are 5 pieces of paper. Which piece^ of paper is both 
black and square? 

A. 1 

b; 2 

C. 3 

• D. 4 

E. 5 , ^ . ' * 

110. Here is a baseball on a sloping board. When the Jball is 
released » what direction yill the ball go? 

A. up the board 

B. down the board , 

C. stay where it is irelea^ed 
^ D. will rise in the /air 

,E. both A and B (meaning both up and down the board) 



DO NOT TURN THE PAGE UNTIL THE TEACHER TELLS YOU TO DO SO 
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1. Thlg Is' a picture of 5 pieces of paper. Which statement '-tdfiatlfiea- . 
all the differences ? 

A. 4 Is a different color 
^ ' B. 2 Is smaller • . 

^ C. 2 Is smaller than all. the others and 4 Is a different 
color 

D. 1, 3, 4, and 5' are the same size 
^ E. 4 and '2 are different from each other v.. 

2. This is a picture, of 8 pieces of paper. If Vou group them by shape, 
^ wtiat is the smallest number of groilps you can make ? 



ERLC 



A, 


1 


-B. 


2 


c. 


3 


D. 


4- 


E. 


5 



3. Here 'are 7 toy airplanes. Airplanes 1, 2, 4, and 6 make up a 
special group. What does this group include^ ? 

A. The plane9 that are, modern jets ^ 

B. The planes that are not' black and are modern jets 

C. The^ planes that are black 

D. The planes that are not black 

E. The praxes that are gray and white old-time 2-wing 
airplaned 

4. This is a picture of pieces of paper which were left in the sun 
for different numbers of days. Which is the only thing you can 
say for sure, based on ^ what you see in the picture ? 

' A. Sample 1 faded more' than sample 2 

B. All p&per will continu^ to fade forever the longer they 
are left in the sun <^ 

C. Any paper l4f t in the sun will fade 

D. Both samples. faded more by day 5 than it had by day 2 

E. Paper will fade in the sun, but cloth will Wot 

5. This is a picture of 5 things. . Which of them has volume ? 

■A. 1 

B. .2 

C. 3 • 



E. 5 
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6. There are two pots of water on a stove. Which choice is the 
* best way of telling how they, are different? 

A. The waternts boiling in pot B . 

B. The gas in on in pot B ♦ 

C. The water getd hot when the gas is on 
^D. The water 19^ not boiling in pot A*- 

E. The water is boiling in pd.C p, but it i3 not 

bo'iliog in pot A- ^ 1 

/ 7., Here ^re 5 objects* Which of them is NOT in the Qame state of 

matte r (sol id » liquid, giis) as all the others? 

A. The pencil^ . ^ ! * 

B. yhe watet ' * - ; 
i. Tl\e toy whistle 

. D. The ball 

E. The baads . 

8. This picture has 4 parts. Eact} paxft shows a qx^mpass, a bar 
magnet and a curved magnet. Xn which two parts are thi?ee * 
• things arranged in the samQ way? * 

. , A. 1 and 3 

B. 2 and 4 

C. 1 and 4 , 

. D. 2 a^d 3 ' . I' - 

E. 1 and 2 : 

- A i -. i ^. ,.. r ■ , ■ . ■ J. . . . ... 



0 
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9. Here aire 5 obj ec^t a. Which object^ could paper weights? 

' .' , ' ^ , . ■ * ■ ' 

A. 1, 3, 4 aitd 5 . V ' / ' 

B. 3 and 4 f , ' 
C: 1, 2 and 5 

D. 2 and 5 « ^ " 

E. 2, 4, and 5 / ^ 

10. Here are 6 objectBt Which' dbjects/ c^n be used for carry;lng waterl 

A. 1, 2, 3, 4 and 6 ; - r ^ / 

B. 1, 2, 3, and 6 

C. 1, 2, 3, 4 and 5 

D. 1, 2, 5 and 6 ; ' ^ , 

E. 1, 2, 3 and 5 - . 

11*. Here are 10 marbles ahd 5 cither objects. Which objects can be 
used to carry all 10 marbles at the same time? 

' ~* * 

- •» ■ ♦ 

A. 1, 3 and 5 

ff. 1 ahd 5 „ ' 
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. .. ' , ^ .'■ '\ 

\ \ ' . , ] 

C. 1, 3 and A ' 

D. 1,- 2 and 3 

E. 2, 3, 4. and 5 ., 

■ ■ / . ■ ■ ' ■ ■ ■ 

12. Which temperature reading Is 25 degrees lover than IS'' Fahrenheit? 

0 V 

. . A. -10^ Fahrenheit . ' 

B. 15^ Fahrenheit. . 

C. - 25^ Fahrenheit ■ ' 

D. 0'' Fahrenheit 

E. AO"! Fahrenheit 

13i ^ This Is la ch^rt of Information about 5 planets. Which of these 
planets nas xhe longest' y6ar? . 

A. /siuplter , ^ 

B. / Skturn^ . . 

C. M^rs 

p. Mercury \ 
E. Uranus 

14. Look at the chart again. ^ Which 2 planets have about the same 
length of day? • ' - 

A. Jupiter and Saturn . • ' 

B. Mars and Jupiter ^ • 

C. Mars and Uranus 

B. Mercury and Uranus " 

E. No two . . 

' * . ■ ♦ T 

15. ^„Thls Is a graph of the boiling temperatures of ^6 different liquids. 

Which liquid has the lowest boiling temperature? 

A. Liquid 1 

B. Liquid 2 

C. Liquid 3 . * ^ 
^ D. Liquid 5 • 

E. Liquid 6 . . 

16. Look at. the graph, again. Which liquids have the same boiling 
temperature? 



A. 4 and 6 

B. 3 and. A 

C. 1 and 5 

D. 1 and 2 
£. '3 and 5 

.1' 
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17* Here is a ball. Whldh of these would be best for measuring the 
. distance around tl^is ball? * 

A. . Tap^ measure ♦ * ^ "\ » - ^ J 

B. . . Mefcer stick * ; . -« 

C. Yard stick , /[ \ 
» D, 1-foot ruler 

6-inch ruler . ^ < ' 

f» • ■ . 

' ^ ■ ■ ""^^^ 

18. Here are 2 clocks. In picture A, i£ is 3:40 in the aftexinoon. 
In picture B it is 6:10 that evening. How much later was ^ 
picture B taken? 

A. 2 hours and 30 minutes - 

B. 6 hours and 10 minutes ^ v 

^ * C. 3 "hours and .40 minutes ' ' ' 

D. 9 hours and "50 minutes 

E. 9. hours and 30 minutes * . 

it. Which unit is used in expressing area? * . . 

A. Jnch ' - ' " / V . 

B. ^"^ CuVic centimeter 

C. Yard / 

-D. Square yard 

^ E. Meter " . - ' ' 5 

20. Which unit is used in measuring weight? . 

r 

A. Gram , 

B. Kilometer ' " . ' - 

C. Pubic Centimeter <y 

D. V Centimeter 

^ E. Neter >• . 
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SCIENCE PROCESSES 
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21. This is a picture of box with its measurements shown on it. 
What is the area of-tk^top of the box?; 

> . . , " ■ i 

A. 20 square inches 

• B. 30b square;lnciies\ ^ 

\^ , C. 35 cubic inches' 

. ' D. .160 square inches 

E^. 35 square ijiches 

22. This is a marble and ^^ruier. If the marble roHs from point A 
to point B in 2 seconds at a steady^speed,« how fast is it going? 

A. 12 iiithes per 2 3econds 
B*; 24 inches per s'ef;ond " ^ 

C. 2 feet per second ; - 

D. 1/2 foot* per second 

E» vlN.f oot per second . , > 

23. This as a picture 9.f a box arid 5 drawings* Which is th^ best 
dravTlng of the box? - . ; . 

: ' ' ^ ^ ' ■ ; ' ' - l"'^* 

\- B. 2 ■ . \ ••■„•- / .. . " i . 

G. 3.. , ■ ' ' ' . ■ 

E. 5 • " .. .' ' 

■ ■ \ ■ . . ^ ^ ■' .' , . •■ ; • ' , 

2A. Which unit is used in measuring .l^ngth?-^ . 

Centimeter * & ^ . - - 

B: -^Gram, " , ■ • -.^ ■ 

C. , Square yard * V # 

D. -Acre . ' ^ - ' ! . 

E. Quart . ' ■ - \ 

25. This is a nfap,^ How far is it from ^&ct\i Town to Birch Fall9? 

A. 9 miles- . • ~- ■ 

B. '18 milSs . - 

C. ' 4 1// miles , '< . . 

D. * 27 "miles , s- 

E. 6 '3/4 miles A ' 

26. Look at the map- again. If Vj^u^ Wjsre using the same sca'le to draw 
\ another map, how far apart wo'ufld you plafre two towns which are 5 

miles frop each other? 
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A. \o Inches 

B. 2/5 Inches > , , 

C. ' ' 5 Inchep * / ^ ^ 

D. 15 inchiss . r 

E. 1 foot ' ^ ^ 

In which pair are^the units close.st in size? 

, A. t^oond and kilometer ' 

B. Yard and Oij^ter ' ^ - ^ 

, C. Met^r arid 

D. Gram and llter^ 
, E. Centimeter and fool 

These are two ice Gub6 tray^. One is filled with vety hot water 
and one with cold water. * Mariy people say: "HOT WATER MA^CES ICE 
, CUBES QUICKER THAN COLO WATEFf." Plan an experiment to t^St this, 
Whicrh choice would be .the best st'atement for helping ybu? - 

A. The hotter the water^you start with, the faster it ' - 
' . will /freeze into ' ice cubes. 1, \ 

B. Hot water freezes i^to jice cubes ^fastei^. - o 

C. Hot water freezes at higher temperatures than cold . 
water. ? ; . ' " ' - 

'^^ D. Hot wat^r freezes into ice cubes fasten because it 
turns ^n the refrigerator. 

E. Hat Water make§ steam ~ steam keeps tfhe refrigerator 

gpi^^g- • ' " . - ' ^' 

If you want to test the statement: "HOT WATER MAKES^ICE CUBEiS 
QUICKER THAN COLD WATER," which factor Is the only -0ne you / 
should allow ^to change dur,ing the experiment? . ' ^ 

r " ■ - ' ■>■'"' 

A. The temperature of the water you qse r 

B. > The amount of water in each tray. x 

C. The positio;! of the trays in the free^i^r ^ ■ 

D. The" ref rigerator in which you put the trays 

E. ' The kind of trays you use , , , 
»f? ■" \ ^ • ^ , 

This, is a'graph vOf .the relsults of an experiment. 200^ seeds 

that were 10 year^ old Shd 200 new seeds w^re planted in 

good soil and wat^r^ each day .' ; ^ ^ * <4 ^ 

100 old seeds \vere put in a cool place 
100 old seeds were put^ in ^ warm "place 
100 new seeds were put in a cool place 
100 new seeds were ptit in a warm place 



\ 
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Five things which may affect the.jgrowth of the ^eeds iare: 
water, heat, soli, age, ^nd light. Which of these were 
tested? . ' ' ' 



A. 
B. 
C. 
D. 
E. 



Heat and age only 

Soil, heat and light only 

Heat, soil, age and llght^only 

Water and soil only 

Water an4 age only < 



31; hLok at the graph ^ 
the graph: 



gialn. Here are i^ome things you ^an see 



on 



1. 182 seeds sprouted 

2. 200 seeds, were 10 years old 

3. 200 seeds were new 

A. 200 seeds were. kept warm 
5. 200 seeds were kept cool a 
K 

Which one happened because of all the others? 



A. 
B. 
C. 
D. 
E. 



1 

2 
3 
4 



\ 



32. Here are 5 containers which will be left put in a thundbr storm. 
Which Is the best container to use\ to find out how many Inches 
of rain will fail?. " ' , " , 



A. 
B. 
C. 
D. 
E. 



1 

2' 

3 

4 

5 



33. tlere are 4 screws and 4 magnets. Which statement CANNOT be 
made just fpNd looking at the picture? 

Screws 2 



A. 
B. 
C. 



Screws 2 
Son^e scr 



heir magnets 



and 3 haye big (heads 
and 3 are stlcklhg to 
s are made from a metal which is not 
magiietlc l - ^ " • ' " ' 
All the screws with big heads In 'this plctuire are 
sticking Eo their magnets 1 I 
Screws 1 and 4.areinot sticklijg to' their iijagnets 
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Here Is some string. The manufacturer plalm 
least 100 lbs.. What Is the best way to ch 




will hold at 
this? 



A. Hang a of 75 pounds on thjjS string, and 
^^ep addlnfelf potind weights unt^ It breaks. 

B. Hang a lOO-pound weight on the String' and see If It 
.breaks.' , * . ._-<2^ 

C. Let two lOO^-pound b6yt pu^l on^each end of a piece 
' X)f the string and see^ If it breaks. * 

D. Hang IfOl pounds oA the string and see If It breaks. 

E. Djbubli^ the string andjhang 30 pounds from It l^nd 
aee If It breaks. . . 



r 



35. 



This Is a chart af the change In length of a metal bar as.lt Is 
heated. Wha£ Is tts length at 40*" C? 



A. \ 101 centimeters 

B. 101.3 centimeters 
e. 102 centimeters 

D. 102.3 centimeters 

E. \ 103 centimeters 



,36. book at the chart again. What will the length .of the bar probably 
be at 100° C? 

A. 103.3 centimeters 

B. 104 centimeters 

C^. 104^.3 centimeters ' . 

, p. 105 centimeters / . , • ' 

fe. 105.5 centimeters ^ . . , 

37. What is the 'best way to check the answerlf^p the last^ two 
questions? ^ . . , 

A. Measure the bar at 100^ C. and then graph all the 

- ' ^ numbejrs to check your answers. . , ' 

B. Measure the bar at IM*" C. and then make a gjraph of 
all the numbers to* check your*, answers. ' : 

C. " Put all your answers on thie chart and s^e if they 
' look correct. « ^ 

D» Measure the b^r at least 5 times at other temperature$ 

Md compare what yotik^flijd with yoiQir answers. >^ ' «^ , 
E, ^Jiasure the bar at 4V- C. ^and at 100**. C. "and compare 
la t you find with your answeirs«> 

S% These are! 2 pictures of ' battery; a bulb, a switch, and some 

wires, wthlcn ^s the only thing you can he sure is different 
^'^ between vthe pictures? C ^ ^ ^ 
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A. Th€ bulb was replaced for picture 2. 

^ B. The vires were tightened for picture 2. 

C. The bulb was screwed In for picture 2* 

D. ' The battery was electrically recharged for picture 2. 

E. Electricity is flowing through the bulb in picture 2. 

If you want to prdve that '"NOT ALL THINGS GET BIGGER AS YOU HEAT 
THEM/' what would you need to do? 

i ' * ■ ' * 

A. Find dne^ thing that does not get blggeir when it , Is 
' ^ heated. V v' ' - 

« b/ Find' all the things that do not get bigger when 
they are heated/ 
C. Fin4 one thing that get^'bigget' when^it 1^ heated. 
D^, Find all t^e; things that get ^bigger when they are 

heated. ' * - 

E. Find all the things th&t do not change size when they 
are heated. ' 

If you want to make this statement r "THE tjOLDER A CITY IS^ THE 
MORE SNOW IT HAS/* what do you need to know about some cities? 

A. ^ The average temperature of each city and the number 
^ of snow plows each has " 
^ B# Tfie number of days school was closed lii each city 

\ bec^use of snow „ * 

C. The average temperature and precipitation of each 
city 

\ D. The average temperature and average snowfall of each 
cfity^ > 
E.XThe average number of times it snows in each city 




\ 
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TV TEST OF SCIENCE PROCESSES 
TEACHERS S TEST MANUAL 
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Introduction 

This test is a modification of Dr. Robert Sh^r Tannenbaum's 
Test of Science Processes . , It has been- adapted for television 
presentation for the intermediate grades in an effort to asseSs the 
scientific skills and abilities that are emphasized la many of the 
modern science prog^ms. pt. Tahnenbaum defines processes and the 
function of the as: 




"Processes**/ are w^s of doing things. For example^ scientists 

at jthings very carefully ^and tell what they 
to be ab^ to measure ^and use numbers. And, 
able to plan and understand. experiments. This 
students can do some of the things scientists 



have to be abld to lo 
^e^. Scientists havi 
scientists have to 
is a test of 



have to do. It is NOT a test of how many acts they know about sentence. 



Alldw sufficient Time before the scheduled TV presentation to 
prepare thK. students for the TV test. Following the introduction to 
Part I of thfe\test, there will be two practice questfbns to show your 
students the t(^st format follpwed immediately by the test questions. 
Be sure your studitvt;s have the necessary materials prior to the TV . 
presentation. They sho^ild have a pencil, -a piece of ^ scratch paper for 
doing any computations, ah::^nswer sheet, and a test booklet. 
Additional pencils and scratbh paper should be accessible to the 
students. DO NOT allow the students to open the test booklet prior to 
the TV 'presentation. DO NOT ••pritoe" the students in any way tJirough 
your knowledge of the test questions as this will negate the te6t , 
results. To help you regulate the^time arid reduce the chancfe x>f a 
period of Inactivity prior to the TV presentation, you nan have 
students fill in *the necessary student Identl&^ation information on 
the answer sheet until the presentation begln^^nd complete the 
Information after;j>the presentation. 



TV Presentation 

Inflormatlon regarding the time (bf the TV presentation of the 
TelevlsJon Test of Science Processes will be given to you by yoin: 
school district. Th^TV presentation will consist of two 30-minute 
programs given one weelfc^^trt. Part-I^wlll Include two practice 

\ " . 
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questions and questions assessing the processes of classifying, 
observing^ comparing, and quantifying. Part II will not repeab any 
•of the practice questions dr directions, but will Include questions 
•assessing the-^processes of ^'measuring. Inferring, experimenting and 
Ipredlctlag. The programs, will present aii the Information necessary^ 
for the questions, Incluqing the visual presentation as well a)^ the 
audio presentation of th^ narration glven| In the test booklet. 

Test Booklet * ^ 

Inform students thid they should . not jwrlte In thd test booklets i ^ 
The test booklets contain the test qjjestlons and answer choices that 
are contained in the narration of the TV presentation. WhiXet is 
no visual information on which to base answers, Xt is inipe^i^atlve that 
the students should not review^any of the test questions prior to the 
^formal i>resentation. This additional "cue" facX9r could invalidate 
the test Results by introducing a reactive effect of Interactloti 
effect. 



Answer Sheet 

The answer j^hee\. that is ^supplied is a machine-scored type. 
Irfstruccthe students )in the correct*^ way to mark answera. The - 
following diagrim shows the correct way to .fill out the name^ bl^ck^ 
Answer -spaces 1J3, 14, 19 and 20 are done Incorrectly as an exaipple of 
common mistakes often encountered with those not Instructed in the 
use of this type of answer sheet. 

Be sut^\lp telP your students to choose only one answer f or^ each 
question and fill in the ansyer space with dark^marksv if a * 
st^udent shouMN^hange an" answer, remind hinf to erase his flr^t mark 
completely, /tf breaks a pencil, he Is^ to hold up hl$ hand and 
the teacher j^ill giye him a new one immediately. . 

ANSWER SHEET PRESENTED) 

*' ~ ' ' ■ . * 

Procedure / 

Thel/e will b^a single answer she^t) for the two programs. Be 
suire to u^e a stud^ht lde^tlf location nqmbef: of identify the/studen^s^ 
Collect t)ie^apswer sheetts and test booklets after each presentation 
and storel^i in a safe place until they are requested. 



William M. Bamhart 
Executive Secretary 



^ ALLEGHENY 
EDUCATIONAL 

' BROADdAST 
COUNCIL, INC 



444 E. College Avefiue 
State College, PA 
(814) 865-3467 



,Date: August 14, 1975 , 

To: Superintendents, Member- Districts o^ AEBC 
^ AEBC District Designates,. 

FromV William M. Bamhart 



th^ Allegheny Educational Broadcast Council is consistently seeking new 
ways of incorporating television into tbe total* instructional system and 
gathering evidence to determine JLts appropriate use in the classroom. 
An opportunity is now available to experiment through television, that 
we have not yet attempted, launchitig us on a long-range evaluation of 
the effectiveness of television as an instructional resource in our 
schoc^s. - 

During the past Jfew years, the Department of Education' (PDE) has 
sponsored a concentrated thrust to use instructional television to 
assist elementary school teachers in the Reaching of -science. The 
materials and related television programa associated with this thrust, 
entitled SCIENCE FOR THE SEVENTIES (SFTS), are part; of a comprehensiye 
effort to assist School districts and individual teachers to itnpfbve 
their science ihstt'uctional programs.* These materials are designed to 
help teachers make a transition to a contemporary science instructional 
program or, if they are already involved in a modern program, to help 
them with instructional strategies and procedures that will increase 
their teaching effectiveness.*^ 

A^Tes^t of Science Processes for which reliability and validity has been 
established, atid which can be administered t?hrough television, has been 
developed by the SFTS staff. This test is designed to be administered^/ 
^'t about the fifth 'grade l^vel in two half-hou.r television segments 
All instructions are given during tht television broadcast^i. ^ Th^ 
person monitoring 'the test need .onlly distribute d test booklet, an 
answer sheet, >nd a H pencil, llto me students, turn on the tjelevision. 



supervise t!he ^group ^or a half hour!, aird then colllect the matjeria 
both' half-hour segmerJts of the teist 



Thje test^is , scheduled to be broadcast • twice during the week of 
September 15, as follpws: ' ' * - 



s for^ 



Part I - 9:30 a.m., September 16 
Part II - 9:30 a.m., September 17 



Part I - 2:00 p.m. , September' 18 * < * 

Part II - 2:00 p.m;, September 19 

If you wlkh to videotape the test an^ offer It another time, you are 
welcome to do so. 

We perceive that this test offers us the opportunity to accomplish a 
number of things: (1) We can explore the feasibility of testing a large 
number of students ovei: a, large geographic area through televisldn/ 

(2) It provides an opportunity to gather some baseline data* on how 
students are performing In this particular area of science ieajaing 
early In the SPTS program. This data can be compared with data ^ 
gathered ln>^future years to determine If growth Is taking place. 

(3) Data can be forwarded to you relative to the performance of your 
schopl district. , ' ^ - . . 

Of course, these test scores must be handlj&d very confidentially. There- 
fore, all scores would remain anonymous In the final report; However, 
"if a school district Vants the composite scores and the general test / 
, norms, this Information would be provided upon request ^by the - 

Superintendent. ^ " i 

♦ ■ •» - 

^ # " > " •> . 

If you are interested in participating, please determine the itumber of*- 
f if th^rade students who will take the test and notify Bill Willfams of 
this number by SEPTEMBER 8. The , appropriate number of- test booklet$ and- 
answer sheets will be forwarded to you* Upon completion of the test, 
return the booklets and answer sheets to AEBC. ^ 

If you wish to participate in «the testing program but are unable~to^ 
meet t\\e scheduled broadcast of the test, AEBC will accept tests for 
scoring until November 15. ^ . 

Further ques>ions 'about SCIENCE FOR THE SEVENTIES and the Test of Science 
Processes will be answered at the AEBC Utll4!zation Conference on 
August 26, or you may contact Bill Williams at the AEBQ office. ? ^ 

WMB/MMS . . i • ' ' ' ■ ' 



APPENDIX D V ' 

LETTER FROM DAVID TORRENCE, TEST DEVELOPER,* 
TELEVISION TEST OF "SCIENCE PROCESSES. TO AREA 
TEST COORDINATORS, DATED DECEMBER 15, 0,975 



THE PENNSYLVANIA STATE:- UNIVERSITY 
University Park, Pennsylvania 16802 



COLLEGE OF EDUCATION 
Division of Academic 
Curriculum and 
Instruction 



December 15, 1975 



r ' 



Dear 



thje 




Recently you exposed- your 5th grade students to the Science fdjr^* 
Seventies' Television Testr of Science Processes,. . We^ have tabu'- >• 



lated the norms for an extensive' sample (in excess of 4,060) throughbut 
the State. \ In an effort to aid you In making appropriate ihtet^r'et2(t^^ 
artd in aiding us to provide' appropriate crye<Ht and to provide you „ , ..« 
additional interprjative service,/I would appreciate your helpxi.n i 
amassing additional Information. ^ / " . - - -^l ■. I 



WojLild you please prdlnide me with a brief (one or ; two para^^ 
^description of your school distrlLfct. /Pleatse include a «3tat^meat\ of ^ 
whether you have g.ro«pS^ heterog^aeously qr homogeneously T i^^: ¥9^^ 
describe your district &s predominantly rpral, subut.ban; or' u^^ 
you a^cribfe to^a moderU science 'approach aS Opposed: (;o 
approavih; and any special characteristics of your :distr^^ 
be noted. -This information wgufci be rautuXlly beQeJxti^ ' f\, \ 



1 hdip^to finalize the test-norms document upar^^eceipt bjE^^^^ 
--^nna\ f rem th^^various school6>-cind will sepd \you^ tfi^e results - 
^1 will avail myself to you ^anTi. yoar cctllea^UdS; pn any/ 



descr%)tions^f rom th^^ various school6>-cind will seod ^yOu?- t^fe 
shortly. Alst), 

manner relative to the "test and its interpretation 
in the eod^avqf; is greatly aj)preciated. 



Your cdop-eration 



cc)X Bobr&lli 
' -Bill. Williams 
^fe^yl- W^J.iver 



Sinc(5reiy , • > ^ 

yo.rtence 
Test Developer 
television" Test of 
ScieQCjs Processes 
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THE {PENNSYLVANIA STATE UNIVERSITY. 
University Park, Pennsylvania 16802 



COLLEGE OF EDUCAtlON 
Division of Academic 
Curriculum and ' . 
Instruction 



Enclosed Is the computer print-out o£ your dist;rict's scores on ' 
the Television TegX lof Science Processes and a sulmuary of the N orming 
Data. I hope tlji^' inf orma\tion is beneficial to your total testing 
program.- u , 

I would like to thank you for participafjting in this worth- 
while television resource. A goal /Of Science for the Seventies 
ie to provide a wide'v^iety of resour^^es and techniques to 
liistructors throughout^he ^tate. We believe that the Television Test 
o f Science Processes, is just one of the ways we can be of service to 
Pennsylvania's educators. 

As a result ^f the teacher's survey ^which accompanied the 
Teievisipn Test of Science Processes, a great deal of inciting 
information was amassed. I would like to address several comments 
which" were repetitive among ;the teacher's remarks. 

n • ' " 

One cotranent that was expressed several times was that the test 
material seemed too di/ficult for fifth grade students at the 
Ijeginning of the school year.'^ A study conducted earlier in 1975 
suggests,,, that the teacherfe' concerns are somewhat justified. While 
the present test results are valid, the study found a significant; : 
difference betw^en^. the: means of an experimental 5th grade' grOMjp who 
-took the test las the end of /the fifth grac^e' experience (April) and , 
the m^ans of the norming Sth grade group who tqok the test| at the'" 
beginning of the fifth grkde experience (September)u 

jWhile alternate tilypotheses may explain this, ,a bl^usible 
hypotliesis ia I that* significant changes in students^ p^rf ormaniie^ in j 
process abilllties, either learned or develoi)°fed, take place as a result 
of -the 5th grade experience This information may help to establish - 



lis. 



optimum time for a tfestitis pragram based on local needs- Any 
* comparisiph :t6 this data to sublsequent testing must be made in the 

same time fr^me.' ' If your district has special needs relative to 
v-ptogram timing, we" would welcome your suggestions. 

" ' ■ ■ ' • ' » : , " 

A seco-nd comment which was expressed by several teachers dealt 
wich thd readability of the question /:ontent; First was the 
suggestion that all answers' be read- aloud for . non-readers , arid second 
■ waa^that' some answers and digtracters were not comprehended by. fifth 
ciders. A p^ilStJstudy found th?it narrating the answers proved -more • 
Jnfiisi^g than not narrating the answers ^- For. "this reason, the • 
Jerbal. c6nte;ot was designed to be at the low third-grade level The 
■question content syntactically was . found to be -at the high second-grade 
reading«alevel.' One aspect of testing that is difficult to control and 
apaears "to b^a a reading problem" in sole . areas is that of cultural bias. 
Question "la, for- example; shows' an array of.dbjects which include the 
"dbjec^" struck in a badm'inton game, , In several districts it was 
easily identified as a "shuttlecock." In many' areas ^ it was recog- 
nized as a "badSitten birdie,'" whil^ in many other areas, it- was 
unrecognizable as any object within the student ' s ,experien<lB. AU test 
d-evelopers^ate faced with" Chisjproblem. It is not a j?eading. problem in 
'the'' pure sense but its recognition is highly correlated with reading^ 
"ability, - The point-biserial correlation of, this itgm, .for example^ 
' suggests tWt fhis doesn' t threaten the validity of , the tgst results, 
aitljough as a teacher I can empathize with the teachers' concerns, - 

x^till another area of some concern is one over which we have no 
" control. This concern is' the viewing . clarity and size of the, television- 
image." Let me^assure you that-all the items were engineered to maximize 
the video signal. The tapes used for the production were of broa<icast 
quality and designed for optimum viewing transmission.- Several 
"Comments' concerned distortioji, vignetting (the truncating of the visual) , 
^~^all size, etc. Th2se problems rest at t^ie^receiver and not With the 
transmission. As this is a "television -first" for many areas, these 
problem&ould bfe add/fessed to the media, services of 'the district.-. 
As future televised resources become more numerous, teachers should / 
worktlosely with their media coordinators to maximize this valuable 
■ aide.- Any help I can offer to this end, I will gladly extend.- 

* ' • ^ ■ 

Overall, the test was well received as indicated. by the s/rvey, 
The informatipn can |be u^ed "Vith confidence- , Additional data/ 
concerning the test will be disdussed with interested parj:i^ 4:p the 
^ extent confiaentiallty permits. X, again, thank you for partiqlpating 
. in thij? rfforTT^d hope to of fer^ Vou ^ continued , service in* this 
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cc/. Bolj Hill 

Bill William^ 
Paul Welliver 




Sincerely, 



David R* Tortence 
Test Coordinator 



